Increasing accuracy of discrete curve transform estimates for curve matching in four or more dimensions

ABSTRACT

System and method for estimating a rotational shift between a first discrete curve and a second discrete curve, where the second discrete curve is a rotationally shifted version of the first discrete curve. First and second discrete curves are received. A rotational shift between the first discrete curve and the second discrete curve is estimated based on the first discrete curve and the second discrete curve. A cumulative rotational shift is updated based on the estimated rotational shift. A rotationally shifted version of the second discrete curve is generated based on the cumulative rotational shift. The estimating, updating, and generating are performed in an iterative manner using the respective rotationally shifted discrete curve for each iteration until a stopping condition occurs, thereby determining a final estimate of the rotational shift between the first discrete curve and the second discrete curve. The final estimate may be used to perform curve matching.

CONTINUATION DATA

This application is a Continuation-in-Part of U.S. patent applicationSer. No. 11/851,477, titled “Increasing Accuracy of Discrete CurveTransform Estimates for Curve Matching in Higher Dimensions”, filed Sep.7, 2007, now U.S. Pat. No. 7,450,765, which is a Continuation of U.S.patent application Ser. No. 10/430,546, titled “Increasing Accuracy ofDiscrete Curve Transform Estimates for Curve Matching”, filed May 6,2003, now U.S. Pat. No. 7,327,887, which is a Continuation-In-Part ofU.S. patent application Ser. No. 10/263,560, titled “Pattern MatchingSystem Utilizing Discrete Curve Matching with a Mapping Operator”, filedon Oct. 3, 2002, now U.S. Pat. No. 7,717,048, which claims priority toU.S. Provisional Application Ser. No. 60/371,474 titled “PatternMatching System Utilizing Discrete Curve Matching with a MappingOperator”, filed Apr. 10, 2002, and which are all hereby incorporated byreference in their entirety as though fully and completely set forthherein.

FIELD OF THE INVENTION

The present invention relates to the field of geometric patternmatching, and more specifically to a system and method for estimatingrotational shifts between discrete curves. For example, the inventionmay relate to locating portions of a target data set that match anobject of interest, e.g., in a template data set, with respect tovarious boundary information, e.g., luminance, color, etc.

DESCRIPTION OF THE RELATED ART

In many applications it is necessary or desired to determine thepresence of an object of interest in a data set, such as a target image.For example, in many image processing applications it is desirable tofind one or more matches of a template image in a larger target image.Exemplary machine vision applications include process monitoring,feedback control, and laboratory automation; image and videocompression; and jitter compensation in video cameras, among others.Various characteristics may be used in classifying a location in thetarget image as a match, including luminance pattern information, colorpattern information, and color information.

Additionally, the object of interest in the target data set or image maybe transformed relative to the known object information, e.g., in thetemplate data set or image. For example, the object of interest in thetarget image may be shifted, scaled, rotated, or may have othergeometric or topological transformations.

Prior art pattern recognition systems have typically used a templatematching technique wherein the stored image or pattern to be located isiteratively compared with various portions of a target image in which itis desired to locate the template. FIG. 1 illustrates the patternmatching problem as known in the prior art. As shown, the patternmatching problem involves a template image, wherein one or moreinstances of the template image are desired to be located in the targetimage. The template image and the target image are provided to a patternmatching algorithm which performs the pattern matching. The patternmatching algorithm generally operates to compare the pixels, e.g., thegrayscale value associated with the pixels, in the template image, or aselected subset of sample pixels, against each of the possible variouslocations in the target image.

Typically, the pattern matching algorithm involves comparing thetemplate image, or a subset of sample pixels representing the templateimage, against locations in the target image on a horizontal pixelcolumn basis and horizontal scan line basis. In other words, the samplepixels representing the template image are compared against a portion ofthe pixels in the target image, such as by using a 2D correlation, thesample pixels representing the template are then moved down or across aone pixel scan line or one pixel column in the target image, and thepattern matching algorithm is repeated, etc. Thus, the pattern matchingalgorithm generally involves comparing the template image pixels againstall possible locations in the target image in an iterative fashion. Thepattern matching may produce the location of the match in the image, thequality of match and possibly the orientation, size and/or scaling ofthe match.

The template is typically compared with portions of the target image byutilizing a correlation based pattern matching, i.e., using normalizedtwo dimensional correlation (normalized 2D correlation). This 2Dcorrelation is performed by placing the template over the respectiveportion of the image and performing a complete normalized 2D correlationbetween the pixels in the template and the pixels in the correspondingportion of the image, using values associated with the pixels, such asgrayscale values. This correlation generally produces a correlationvalue which indicates the degree of correlation or match. For example,the correlation value may range between −1 and +1, wherein +1 indicatesa complete match, 0 indicates no match, i.e., that the two images areuncorrelated, and −1 indicates that the two images are anti-correlated,i.e., a complete reversal of a match.

The normalized 2D correlation operation is based on a point-wisemultiplication wherein the template is first conceptually placed over aportion of the image, the value associated with each point or pixel ofthe template is multiplied with the corresponding pixel value in therespective portion of the target image, and the result is summed overthe entire template. Also, as noted above, the template image isgenerally compared with each possible portion of the target image in aniterative fashion. This approach is thus very computationally intensive.

Various optimizations or algorithms have been developed to provide amore efficient pattern matching technique. One prior art technique is touse selected samples or pixels from the template image, referred to assample pixels, to represent the template image and hence to reduce thenumber of computations in the correlation.

FIG. 2 illustrates the pattern matching process of the prior art whichinvolves characterization of the template with a reduced number ofsample pixels. In this process, a characterization of the template isperformed to extract features from the template image. In other words,the template is characterized to represent the template image with alesser number of points or pixels, referred to as sample pixels, whichpresumably accurately characterize the template image. The templateimage is characterized in this fashion because the time required for thepattern matching is generally directly proportional to the number ofpoints or pixels representing the template image which are used in thepattern matching. Thus the template is characterized to reduce thenumber of samples or pixels which are used in the correlation operation,thereby reducing the amount of computation. Once a lesser number ofsample pixels have been generated, these sample pixels are then used inthe pattern matching algorithm to locate instances of the template imagein the target image.

Another prior art technique for performing pattern matching utilizes hueplane or color information, either alone or in combination with patternmatching. Utilizing color information can often be used to simplify agrayscale pattern matching problem, e.g., due to improved contrast orseparation of an object from the background. Also, some applications mayutilize color information alone, i.e., not in conjunction with patterninformation, to locate target image matches, e.g., for cases when anapplication depends on the cumulative color information in a region andnot how the colors are arranged within the region or the spatialorientation of the region.

In machine vision applications, color is a powerful descriptor thatoften simplifies object identification and extraction from a scene.Color characterization, location, and comparison is an important part ofmachine vision and is used in a large class of assembly and packaginginspection applications. Inspection involves verifying that the correctcomponents are present in the correct locations. For example, colorinformation may be used in inspecting printed circuit boards containinga variety of components; including diodes, resistors, integratedcircuits, and capacitors. These components are usually placed on acircuit board using automatic equipment, and a machine vision system isuseful to verify that all components have been placed in the appropriatepositions.

As another example, color information is widely used in the automotiveindustry to verify the presence of correct components in automotiveassemblies. Components in these assemblies are very often multicolored.For example, color characterization may be used to characterize andinspect fuses in junction boxes, i.e., to determine whether all fusesare present and in the correct locations. As another example, it isoften necessary to match a fabric in one part of a multi-colorautomobile interior. A color characterization method may be used todetermine which of several fabrics is being used.

Another prior art technique for performing pattern matching is referredto as geometric pattern matching, which may also be referred to as curvematching or shape matching. Geometric pattern matching generally refersto the detection and use of geometric features in an image, such asboundaries, edges, lines, etc., to locate geometrically defined objectsin the image. The geometric features in an image may be reflected invarious components of the image data, including, for example, luminance(gray-scale intensity), hue (color), and/or saturation. Typically,geometric features are defined by boundaries where image data changes,e.g., where two differently colored regions abut. Geometric patternmatching techniques are often required to detect an object regardless ofscaling, translation, and/or rotation of the object with respect to thetemplate image. For further information on shape or geometric patternmatching, see “State-of-the-Art in Shape Matching” by Remco C. Veltkampand Michiel Hagedoorn (1999), and “A Survey of Shape AnalysisTechniques” by Sven Loncaric, which are both incorporated herein byreference.

An issue that arises in many pattern matching applications is that thetarget image being analyzed may contain other objects besides the objectof interest, i.e., the template image. The presence of these otherobjects may complicate the pattern matching process, in that the otherobjects may differ from the template image to various degrees, with someof the objects being fairly similar, and others being quite differentfrom the template image. For a simplified example, assume an applicationwhere the template image is a square, and the target image contains asquare, two rectangles, and a circle. To reliably match/detect thesquare in the target image, a pattern matching technique must not onlydistinguish between the square and the circle, but also between thesquare and each of the rectangles. However, in many cases, a patternmatching algorithm which successfully distinguishes a square from acircle may not reliably distinguish a square from a rectangle.Conversely, a pattern matching algorithm which successfullydistinguishes a square from a rectangle may not reliably distinguish asquare from a circle. Thus, when the target image presents a complexscene, the task of distinguishing between the object of interest andobjects not of interest may become difficult. This issue becomesincreasingly important when pattern matching is performed with ‘realworld’ image data, i.e., when the target image is not pre-processed tofacilitate pattern matching with the particular pattern matchingalgorithm used.

In many curve matching applications, two discrete curves, e.g., a firstcurve and a second curve, may differ by a small rotational shift, wherethe second curve is a slightly rotated version of the first curve. Priorart curve matching methods have typically involved cyclically shifting(rotating) the second curve by increments of the angular distancebetween successive points in the curve. However, such “quantized” cyclicshifting may substantially limit the accuracy with which the curves maybe prepared, e.g., aligned or corrected, for pattern matching purposes.

Therefore, improved methods are desired for preparing discrete curvesfor comparison, e.g., for a curve matching application.

SUMMARY OF THE INVENTION

Various embodiments of a system and method for estimating a small shifta between two discrete curves, y and z, are presented, where curve z isa slightly rotated version of curve y. The curves have preferably beennormalized with respect to position and length. In various embodiments,the curves may be 2 dimensional, 3 dimensional, or of higherdimensionality. Each curve preferably includes a sequence of points,where the angular difference between successive points in the discretecurve may be considered an intrinsic angular resolution of the curve. Inone embodiment, the curves may each be represented by a respectivesequence of points in the complex plane. In other embodiments, thecurves may comprise respective point sequences in 3D or higherdimensions.

A current estimate may be made of a rotational shift between the firstand second discrete curves, and may be used to update a cumulativerotational shift. A rotationally shifted version of z may be calculatedusing the computed value of the estimated rotational shift. In oneembodiment, rotationally shifting the curve z may include interpolatingthe curve. Any of various interpolation methods may be used, butquadratic or spline interpolations generally preserve the original shapebetter than linear interpolations and so may be used in preferredembodiments. Note that the new version of z may be computed by rotatingthe original curve z by the updated cumulative rotational shift, oralternatively, may be computed by rotating the previous version of z bythe current estimate. The former approach is preferably used to minimizeaccumulated numerical errors.

A determination may be then be made as to whether a stopping conditionoccurs. If a stopping condition is not detected, then the method mayrepeat iteratively, calculating a new estimate using the rotationallyshifted version of z, performing the above steps in an iterative manneruntil the stopping condition obtains. Note that since the new estimateis based on the rotationally shifted version of z, the magnitude of thenew value will generally be less than that of the current (i.e.,previous) value, and thus, as each successive computation of theestimate is made, the values will tend toward zero, although theprogression may include both positive and negative values.

In one embodiment, the stopping condition may occur when the magnitudeof the estimated rotational shift is less than some specified thresholdvalue (e.g., epsilon, representing a sufficient level of accuracy),typically after only a few iterations (possibly even just one).Alternatively, in another embodiment, the stopping condition may occurwhen a certain number of iterations have elapsed. Of course, any otherstopping condition may be used as desired.

If a stopping condition is determined to occur, then results may beoutput, e.g., a final cumulative estimated rotational shift may beoutput, e.g., to a display device, to a file, to a process, to anexternal system, and so forth, where the final cumulative estimatedrotational shift is an improved estimate of the rotational differencebetween the two curves y and z.

In one embodiment, the most recent version of curve z (e.g., z^(k),where k is the index value of the last iteration) may be output, e.g.,to a matching process, where curves y and z^(k) may be analyzed todetermine whether or to what degree the curves match.

In one embodiment, the estimated rotational shift may be normalized withrespect to the index, i.e. a value of 1 may stand for a rotation orcyclic shift of exactly 1 index point. It is noted, however, than ingeneral the computed value of the estimate will be substantially lessthan 1, i.e., the value will generally be less than the intrinsicangular resolution of the discrete curves.

In one embodiment, 2D or 3D discrete curves may be augmented withadditional image data to generate discrete curves in higher dimensions.For example, in one application of the method 2D curves may be augmentedwith gray-scale information related to the curves, e.g., to improve theaccuracy in geometric pattern matching. In other words, gray-scaleinformation may be used as an additional dimension, effectively makingthe 2D curves into 3D curves. To this end the shape information (twospatial coordinates in the 2-D case) may be augmented by the gray-scaleinformation (a third spatial coordinate in the 2-D case) at each point.Thus, the gray-scale information may be interpreted as an extradimension in the curves. The method described above may then be used todetermine sub-angle rotational differences between the two (originally2D) curves.

It should be noted that this approach is also applicable in even higherdimensions, where, for example, other additional image data may beincluded as corresponding additional dimensions to be added to curvesthat are originally 2D or 3D (or of even higher dimensionality). Forexample, where the first discrete curve and the second discrete curveeach comprise a respective sequence of points, each point may include arespective two or more spatial coordinates, and a respective one or moreadditional coordinates, where each additional coordinate is based onrespective additional image data associated with the point. Each of theone or more additional coordinates may then be interpreted as additionalspatial coordinates for the point. Thus, the effective dimensionality ofthe curves may be increased by including additional image data for eachpoint as additional spatial coordinates or dimensions, where theadditional information may improve the accuracy of curve matching of thetwo curves. For example: a 2D curve augmented by 3D color informationmay result in a 5D representation, or a system or process may berepresented using generalized coordinates of high-dimensionality. Thus,the above techniques may be applied to higher dimensional curves, i.e.,curves of 4 or more dimensions.

Thus, various embodiments of the above methods may be used to determinesub-angle accuracy not only for two- and three-dimensional discretecurves, but for higher-dimensional curves as well, e.g., curves of 4 ormore dimensions, where a second curve is a slightly rotated version of afirst curve. Once the second curve has been rotated in accordance withthe computed final estimate, the curves may be tested or processed todetermine whether and/or to what extent they match.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description of the preferred embodiment is consideredin conjunction with the following drawings, in which:

FIG. 1 illustrates pattern matching as performed in prior art;

FIG. 2 illustrates pattern matching performed in prior art, whichincludes characterization of the template image with fewer pixels forreduced computation;

FIG. 3 illustrates a computer system suitable for performing patternmatching according to one embodiment of the present invention;

FIG. 4 illustrates an image acquisition (video capture) system foracquiring images;

FIG. 5 is a high-level block diagram of the image acquisition system;

FIG. 6A is a flowchart diagram illustrating one embodiment of a methodfor re-sampling discrete curves;

FIG. 6B illustrates example results of the method of FIG. 6A;

FIG. 7 is a flowchart diagram illustrating one embodiment of a methodfor locating regions of a target image that match a template image;

FIG. 7A is a flowchart diagram illustrating one embodiment of thelearning phase of the method of FIG. 7;

FIG. 7B is a flowchart diagram illustrating one embodiment of a methodfor determining a set of discrete curves in an image;

FIG. 7C is a flowchart diagram illustrating one embodiment of a methodfor determining the mapping operator for the method of FIG. 7A;

FIG. 7D is a flowchart diagram illustrating one embodiment of a methodfor normalizing discrete curves for the method of FIG. 7A;

FIGS. 8A and 8B are flowchart diagrams illustrating embodiments of thematching stage of the method of FIG. 7;

FIG. 9A is a flowchart diagram illustrating one embodiment of a methodfor matching open discrete curves with shift, scale, and rotationinvariance;

FIG. 9B is a flowchart diagram illustrating another embodiment of amethod for matching open discrete curves with shift, scale, and rotationinvariance;

FIG. 9C illustrates a mapping of a set of open curves and crosscorrelations for the original and mapped discrete curves, according toone embodiment;

FIG. 10A illustrates two closed curves which are counter-clockwiseoriented, according to one embodiment;

FIG. 10B is a flowchart diagram illustrating one embodiment of a methodfor matching a closed discrete curve;

FIG. 10C illustrates correlation between a closed discrete curve andcyclic shifted version of the curve, according to one embodiment;

FIG. 11 is a flowchart diagram illustrating one embodiment of a methodfor matching against a set of closed discrete curves;

FIG. 12A illustrates correlation between a closed discrete curve andcyclic shifted version of the curve, according to another embodiment;

FIG. 12B illustrates correlation between a closed discrete curve andcyclic shifted version of the curve with linearization of angles,according to another embodiment;

FIG. 13 is a flowchart diagram illustrating one embodiment of a methodfor estimating sub-angle accuracy in slightly rotationally shiftedcurves;

FIG. 13A is a more detailed flowchart diagram illustrating oneembodiment of the method of FIG. 13;

FIG. 13B is a detailed flowchart diagram illustrating one embodiment ofa method for estimating sub-angle accuracy in slightly rotationallyshifted 3D curves;

FIG. 14A is a flowchart diagram illustrating one embodiment of a methodfor matching 2D closed discrete curves;

FIG. 14B illustrates correlation between a 2D closed discrete curve andcyclic shifted version of the curve using the method of FIG. 14A,according to another embodiment; and

FIG. 15 is a flowchart diagram illustrating one embodiment of a methodfor matching 3D closed discrete curves.

While the invention is susceptible to various modifications andalternative forms specific embodiments are shown by way of example inthe drawings and are herein described in detail. It should beunderstood, however, that drawings and detailed description thereto arenot intended to limit the invention to the particular form disclosed.But on the contrary the invention is to cover all modifications,equivalents and alternatives falling within the spirit and scope of thepresent invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Incorporation by Reference

The following patent applications are hereby incorporated by referencein their entirety as though fully and completely set forth herein:

U.S. patent application Ser. No. 09/227,506 titled “Pattern MatchingSystem and Method Which Performs Local Stability Analysis for ImprovedEfficiency” filed on Jan. 6, 1999, whose inventors are Dinesh Nair,Lothar Wenzel, Nicolas Vazquez, and Samson DeKey; and

U.S. Provisional Application Ser. No. 60/371,474 titled “PatternMatching System Utilizing Discrete Curve Matching with a MappingOperator”, filed Apr. 10, 2002.

The following publications are hereby incorporated by reference in theirentirety as though fully and completely set forth herein:

The National Instruments IMAQ™ IMAQ Vision Concepts Manual; and

“Efficient Matching Of Discrete Curves”, by Lothar Wenzel, NationalInstruments, Austin, Tex.

Terms

The following is a glossary of terms used in the present application:

Memory Medium—Any of various types of memory devices or storage devices.The term “memory medium” is intended to include an installation medium,e.g., a CD-ROM, floppy disks 104, or tape device; a computer systemmemory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM,Rambus RAM, etc.; or a non-volatile memory such as a magnetic media,e.g., a hard drive, or optical storage. The memory medium may compriseother types of memory as well, or combinations thereof. In addition, thememory medium may be located in a first computer in which the programsare executed, or may be located in a second different computer whichconnects to the first computer over a network, such as the Internet. Inthe latter instance, the second computer may provide programinstructions to the first computer for execution. The term “memorymedium” may include two or more memory mediums which may reside indifferent locations, e.g., in different computers that are connectedover a network.

Carrier Medium—a memory medium as described above, as well as signalssuch as electrical, electromagnetic, or digital signals, conveyed via acommunication medium such as a bus, network and/or a wireless link.

Programmable Hardware Element—includes various types of programmablehardware, reconfigurable hardware, programmable logic, orfield-programmable devices (FPDs), such as one or more FPGAs (FieldProgrammable Gate Arrays), or one or more PLDs (Programmable LogicDevices), such as one or more Simple PLDs (SPLDs) or one or more ComplexPLDs (CPLDs), or other types of programmable hardware. A programmablehardware element may also be referred to as “reconfigurable logic”.

Medium—includes one or more of a memory medium, carrier medium, and/orprogrammable hardware element; encompasses various types of mediums thatcan either store program instructions/data structures or can beconfigured with a hardware configuration program.

FIG. 3—Computer System

FIG. 3 illustrates a computer system 102 which may perform pattern matchlocation according to one embodiment of the present invention. Thecomputer system 102 may comprise one or more processors, a memorymedium, display, and an input device or mechanism, such as a keyboard ormouse, and any other components necessary for a computer system.

The computer system 102 may perform a pattern characterization method ofa template image and may use information determined in this analysis todetermine whether a target image matches the template image and/or tolocate regions of the target image which match the template image, withrespect to pattern information. Images that are to be matched arepreferably stored in the computer memory and/or received by the computerfrom an external device.

The computer system 102 preferably includes one or more softwareprograms operable to perform the pattern match determination and/orlocation. The software programs may be stored in a memory medium of thecomputer system 102. The term “memory medium” is intended to includevarious types of memory, including an installation medium, e.g., aCD-ROM, or floppy disks 104, a computer system memory such as DRAM,SRAM, EDO RAM, Rambus RAM, etc., or a non-volatile memory such as amagnetic medium, e.g., a hard drive, or optical storage. The memorymedium may comprise other types of memory as well, or combinationsthereof. In addition, the memory medium may be located in a firstcomputer in which the programs are executed, or may be located in asecond different computer which connects to the first computer over anetwork. In the latter instance, the second computer may provide theprogram instructions to the first computer for execution.

Various embodiments further include receiving or storing instructionsand/or data implemented in accordance with the foregoing descriptionupon a carrier medium. Suitable carrier media include a memory medium asdescribed above, as well as signals such as electrical, electromagnetic,or digital signals, conveyed via a communication medium such as networksand/or a wireless link.

Also, the computer system 102 may take various forms, including apersonal computer system, mainframe computer system, workstation,network appliance, Internet appliance, personal digital assistant (PDA),television system or other device. In general, the term “computersystem” can be broadly defined to encompass any device having aprocessor which executes instructions from a memory medium.

The software program(s) may be implemented in any of various ways,including procedure-based techniques, component-based techniques,graphical programming techniques, and/or object-oriented techniques,among others. For example, the software program may be implemented usingActiveX controls, C++ objects, Java Beans, Microsoft Foundation Classes(MFC), or other technologies or methodologies, as desired. A CPU, suchas the host CPU, executing code and data from the memory mediumcomprises a means for performing pattern match location according to themethods or flowcharts described below.

FIG. 4—Machine Vision System

FIG. 4 illustrates a machine vision system or image acquisition system,which is an example of one application of the present invention. Thepattern match determination/location techniques described herein may beused in various types of machine vision or motion control applications.For example, the computer 102 may be embodied in various form factorsand/or architectures, e.g., a robot or embedded device, among others. Itis also noted that the pattern match location techniques describedherein may be performed in any of various manners, either in software,programmable logic, or hardware, or a combination thereof.

In the machine vision system of FIG. 4, computer system 102 is coupledto a camera 112 and operates to receive one or more images. The computersystem 102 may be operable to perform a pattern characterization methodto determine pattern information of the template image. In the presentapplication, the term “template image” is used to refer to either anentire image, or a portion of an image, e.g., a region of interest(ROI). The computer system 102 may also be operable to perform a searchof a target image to locate target image regions that match the patterninformation of the template image. As described below, the search may beperformed to locate matching regions with any of various degrees ofexactness, as appropriate for a particular application.

FIG. 5—Image Acquisition System Block Diagram

FIG. 5 is a high-level block diagram of the image acquisition system ofFIG. 4 for acquiring an image for pattern matching according to oneembodiment of the present invention. It is noted that the block diagramof FIG. 5 is exemplary only, and other computer system architectures maybe used as desired. For example, the present invention may beimplemented in a “smart camera”, which integrates a sensor, analog todigital (A/D) converter, CPU, and communications devices together in asingle unit. The present invention may be embodied in otherarchitectures, devices, or embodiments, as desired.

As shown in FIG. 5, the host computer 102 preferably comprises a CPU202, a bus bridge 204, system memory 206, and a peripheral bus 212. TheCPU 202 is coupled to the bus bridge 204. The bus bridge 204 is coupledto the system memory 206 and the CPU 202, and couples to the peripheralbus 212. In the preferred embodiment, the peripheral bus 212 is the PCIexpansion bus, although other types of buses may be used.

In this embodiment, the host computer system 102 also includes a videocapture board 214 which is adapted for coupling to the video source 112.The video capture board 214 is preferably coupled to the peripheral bus212. In addition to the video capture board 214, other peripheraldevices (216 and 218) may be coupled to the peripheral bus 212, such asaudio cards, modems, graphics cards, network cards, etc.

The video source 112 supplies the analog or digital video signals to thevideo capture board 214. The video capture board 214 transfers digitizedvideo frames to the system memory 206 through peripheral bus 212 and busbridge 204. In this embodiment, the video capture board 214 acquires thetarget image and transfers it to system memory 206. One or more regionsof interest (ROI) may be specified in the target image which are desiredto be searched for regions having pattern information that matches thepattern information of a template image, or the entire target image maybe searched.

The system memory 206 may store a template image. The system memory 206may also receive and/or store one or more other images, such as selectedregions of interest (ROIs) in the template image or another image, oracquired target images. The system memory 206 also preferably storessoftware according to the present invention which operates to analyzethe pattern information of the template and target images. The softwaremay also be executable to perform various pattern match locationmethods, as described below. The system memory 206 may store the patterninformation of the template image for comparison to various regions inthe target image during the pattern match location process.

The term “image,” as used herein, may refer to any of various types ofimages. An image may be obtained from any of various sources, includinga memory medium. An image may, for example, be obtained from an imagefile, such as a BMP, TIFF, AIPD, PNG, JPG, or GIF file, or a fileformatted according to another image format. An image may also beobtained from other sources, including a hardware device, such as acamera, framegrabber, scanner, etc. An image may be a complex image, inwhich pixel values (positions) have a real part and an imaginary part.

It is noted that, in a pattern match application, the patterninformation of the template image may be pre-calculated and stored inthe computer, and the actual template image is then not required to bestored or used for subsequent pattern match determination/locationoperations with respective target images. Thus, when a target image isacquired, the software may compare the pattern information of the targetimage with the pre-computed pattern information of the template image.

The present invention is preferably implemented in one or more softwareprograms which are executable by a processor or CPU. The softwareprogram(s) of the present invention are preferably stored in a memorymedium of a computer as described above.

Although many of the embodiments described herein relate to images andimage processing, it is noted that the techniques described are broadlyapplicable to data sets and data processing. In other words, variousembodiments of the invention may be used to perform discrete curvematching where the discrete curves are determined from data as opposedto just images.

Certain concepts related to Sturm-Liouville Theory and curve matchingmay be useful in understanding various embodiments of the presentinvention, and thus are presented below.

Sturm-Liouville Theory

The Sturm-Liouville theory is a well-established mathematical topic withnumerous applications in science and engineering (e.g. Chirikjian andKyatkin [2001]). Some results and methods of this theory are presentedbelow. It should be noted that for brevity, the most general version ofthe theory is not presented.

Let s(x), w(x), and q(x) be real-valued differentiable functions on thereal finite and closed interval [a,b], where s(x) and w(x) are positivefunctions on the given interval. The solutions of differential equation:

$\begin{matrix}{{{\frac{\mathbb{d}}{\mathbb{d}x}\left( {{s(x)}\frac{\mathbb{d}y}{\mathbb{d}x}} \right)} + {\left( {{\lambda\;{W(x)}} - {q(x)}} \right)y}} = 0} & (1)\end{matrix}$in the unknown function y for different eigenvalues λ and appropriateboundary conditions generate sets of orthogonal eigenfunctions y(x). Ithas been shown (Birkhoff and Rota [1960]) that the eigenfunctions ofSturm-Liouville differential equations (1) form a complete set offunctions in L²([a,b],R). Perhaps the most familiar situation is s(x)=1,w(x)=1, and q(x)=0, [a,b]=[0,2π], where the boundary conditions arey(a)=y(b) and y′(a)=y′(b). The differential equation (1) reduces to:

$\begin{matrix}{{{\frac{\mathbb{d}^{2}y}{\mathbb{d}x^{2}} + {\lambda\; y}} = 0}{{y(0)} = {{{y\left( {2\pi} \right)}\mspace{14mu}{and}\mspace{14mu}{y^{\prime}(0)}} = {y^{\prime}\left( {2\pi} \right)}}}} & (2)\end{matrix}$The eigenvalues and real-valued eigenfunctions are:λ=0 and y(x)=1λ=n² and y(x)=sin(nx), y(x)=cos(nx) for n=1,2, . . .  (3)Functions f in L²([a, b], R) can be represented in the L² sense by aseries:

$\begin{matrix}{{{f(x)} = {\sum\limits_{n = 0}^{\infty}{d_{n}{y_{n}(x)}}}}{where}{d_{n} = \frac{\int_{a}^{b}{{\mathbb{d}{{xf}(x)}}{y_{n}(x)}{w(x)}}}{\int_{a}^{b}{{\mathbb{d}{{xy}_{n}(x)}^{2}}{w(x)}}}}} & (4)\end{matrix}$

It is common to normalize functions y_(n)(x), i.e. the denominator in(4) vanishes. The function w is called the weight function. Many of theresulting orthogonal function systems {y_(n)(x)} satisfy three-termrecurrence relations:y _(n+1)(x)=C ₁(x,n)y _(n)(x)+C ₂(x,n)y _(n−1)(x)  (5)

Particularly, in the case when {y_(n)(x)} is a set of polynomials, thefunctions C₁ and C₂ are of the form C₁(x,k)=a_(k)x+b_(k) and C₂(x,k)=c_(k), respectively. All numbers a, b, and c depend only on k. Itcan be shown that the three-term recurrence relation:p _(n)(x)=(a _(n) x+b _(b))p _(n−1)(x)+c _(n) p _(n−2)(x)  (6)generates all possible systems of orthogonal polynomials, where

$\begin{matrix}{{{p_{- 1}(x)} = 0}{{p_{0}(x)} = 1}{{a_{n} = \frac{1}{\left( {p_{n - 1},p_{n - 1}} \right)^{\frac{1}{2}}}},{b_{n} = {- \frac{\left( {{xp}_{n - 1},p_{n - 1}} \right)}{\left( {p_{n - 1},p_{n - 1}} \right)^{\frac{3}{2}}}}},{c_{n} = {- \frac{\left( {p_{n - 1},p_{n - 1}} \right)^{\frac{1}{2}}}{\left( {p_{n - 2},p_{n - 2}} \right)^{\frac{1}{2}}}}}}{\left( {p,q} \right) = {\int_{a}^{b}{{p(x)}{q(x)}{w(x)}{\mathbb{d}x}}}}} & (7)\end{matrix}$

Let p_(k)(x) be a complete system of orthonormal polynomials over[−1,1], where w(x) is a non-negative weight function used to define thescalar product in this space. Let x₀, x₁, . . . , x_(n) be the roots ofp_(n+1), . . . , and let w₀, w₁, . . . , w_(n)w₀ be the solution of(Gaussian quadrature):

$\begin{matrix}{{{\sum\limits_{j = 0}^{n}{x_{j}^{k}w_{j}}} = {{\int_{- 1}^{1}{x^{k}{w(x)}{\mathbb{d}x}\mspace{14mu}{for}\mspace{14mu} k}} = 0}},\ldots\mspace{14mu},n} & (8)\end{matrix}$It can be shown that

$\begin{matrix}{{\sum\limits_{j = 0}^{n}{{f\left( x_{j} \right)}w_{j}}} = {\int_{- 1}^{1}{{f(x)}{w(x)}{\mathbb{d}x}}}} & (9)\end{matrix}$for any polynomial of order not exceeding 2n+1. In particular,

$\begin{matrix}\begin{matrix}{{\sum\limits_{k = 0}^{n}\;{{p_{i}\left( x_{k} \right)}{p_{j}\left( x_{k} \right)}w_{j}}} = {\int_{- 1}^{1}{{p_{i}\left( x_{k} \right)}{p_{j}\left( x_{k} \right)}{w(x)}\ {\mathbb{d}x}}}} \\{{= {\delta_{ij}\mspace{14mu}{for}\mspace{14mu} j}},{k \leq n}}\end{matrix} & (10)\end{matrix}$

Gaussian quadrature provides tools that translate continuousorthonormality relations into discrete ones. Equation (10) can beinterpreted as follows. The zeros of such an orthonormal polynomial(basis function) may be used to efficiently sample curves, as describedbelow with reference to FIG. 6A.

Shape Based Geometrical Description

Shape analysis has many applications in engineering, biology, chemistry,geography, medicine, and imagine processing. General shape spaces arewell understood, e.g. Kendall et al., [1997], Kendall [1977, 1984], Came[1990]. Such spaces are based on specific sets of transformation groupsG that lead to Riemannian metrics and Lie group theoretical approaches.An example is Kendall's shape space

$\sum\limits_{m}^{k}\;$of k points in an m-dimensional Euclidean space where the group G oftransformations consists of translation, rotation, and scaling. Asuitable distance in

$\sum\limits_{m}^{k}\;$is the Riemannian metric ρ. This metric can be defined as follows. LetA_(p) and B_(p) be so-called preshapes of two configurations A and B. Aand B are point sets of same size k in R^(m). Preshapes are normalizedversions of the original shape (centered at 0 and Euclidean norm 1). TheRiemannian metric ρ in shape space is defined as ρ(A,B)=arccos{trace(Λ)}where the matrix Λ is the diagonal m by m matrix with positive elementsgiven by square roots of the eigenvalues of A_(p) ^(T)B_(p)B_(p)^(T)A_(p), except the smallest diagonal element which is negative ifdet(B_(p) ^(T)A_(p))<0. Related distances are full and partialProcrustes distances (see e.g. Kent [1992]). The term ‘full’ stands forminimization over the full set of similarity transforms and the term‘partial’ stands for minimization only for translation and rotation.

The special case of shapes in R² can be interpreted as the study ofpoint sets in the complex plane, C. In doing so it can be shown that thefull and partial Procrustes distances in Kendall's shape space for pointin R² can be obtained as the solution to complex linear regressionproblems of the form:B=rAe ^(iα)+1_(k) c+Ewhere c is the complex location, r is the scale, α the angle ofrotation. After centering and resealing A and B as before symmetricmeasures of residual discrepancy can be derived.d _(full) ²=1−A*BB*Ad _(partial) ²=2(1−∥A*B∥)  (11)The former distance will be used in this paper to develop robust andefficient geometric pattern matching algorithms based on shapes.

In general, matching of shapes in higher dimensional spaces R^(m) cannotbe cast as a problem in linear regression. After pre-shaping two givenconfigurations A and B (centered at 0 and Euclidean norm 1) symmetricresidual discrepancy measures are:d _(m,full) ²=1−trace(Λ)²d _(m,partial) ²=2(1−trace(Λ))  (12)

whereA_(p) ^(T)B_(p)=VΛU^(T)is the singular value decomposition.

There are generalizations of distances to affine spaces and k-tuplets ina differentiable Riemannian manifold where G is a Lie group.

Optimal Discrete Open Curve Descriptors

Depending on the group of transformation, numerous open curvedescriptors can be developed, including shift and scale invariant,shift, scale, and rotation invariant curve descriptors, as describedbelow. In general, curve descriptors may be determined which may provideeffective means for performing shape/pattern matching between discretecurves representing image objects in acquired images, as described belowwith reference to FIGS. 6-13B.

Shift and Scale Invariance

Given a set S={c₁, . . . , c_(M)} of M discrete open and oriented curvesin the plane, where all curves consist of exactly N discrete points. Itis assumed that all curves are equally sampled, i.e. the distancebetween two adjacent points of a given curve is constant. The goal isthe construction of a similarity measure that reflects the fact that allthese curves are perpendicular to each other in a space of discrete opencurves, where shift and scale invariance is achieved. To this end, anon-negative weight vector w=w₁, . . . , w_(N) may be generated thatminimizes (in a certain sense) deviations from the orthonormalityproperty of all curves in S.

Curves c_(i)(i=1, . . . , M) can be regarded as point sets in thecomplex plane C. Because of the desired shift and scale invariance anormalization procedure (mean value equal to zero, sum of lengths of allcomplex numbers in c_(i) is equal to 1) is quite natural. Let all curvesin S={c₁, . . . , c_(M)} be normalized in the described sense. Thedesired orthonormality property results in a linear system:

$\begin{matrix}\begin{matrix}{\left\langle {c_{n},c_{m}} \right\rangle_{w} = {\sum\limits_{i = 1}^{N}\;{c_{ni}c_{mi}^{*}w_{i}}}} \\{= \delta_{nm}}\end{matrix} & (13)\end{matrix}$where the unknowns w₁, . . . , w_(N) are non-negative real numbers (theweight vector). If (13) is replaced with the matrix notation Cw=d, itfollows that(C ^(T))*Cw=(C ^(T))*d  (14)

Both the matrix on the left side and the vector on the right arereal-valued. Thus, expression (13) may provide a set of equations withwhich each W_(m) may be solved. In practical applications, usually thesystem (13) is highly over-determined. Note that there are many moreequations than terms in w, thus the system of equations comprises anover-determined linear system which may be solved via Single ValueDecomposition. Standard methods can be applied to generate the bestsolution in the mean-squared sense in a numerically stable and fastmanner.

Because of the special structure of (13), typically, many components ofthe unknown vector w are positive. Even if this were not the case, thecomputed weight vector w could be used to determine distances betweengiven curves, where these curves are variations of S. To take advantageof the Sturm-Liouville theory, it must be guaranteed that all componentsof w are non-negative. In other words, linear systems with constraintssuch as:w>0 or upper≧w≧lower≧0in conjunction with (14) must be solved. Efficient algorithms for thisclass of problems exist, e.g. Active Set Method or Hildreth-d'EsopoAlgorithm (e.g. Bronshtein and Semendyayev [1997]).FIG. 6A-6B—Re-Sampling of Discrete Curves

In a certain sense, a non-negative weight function w, as describedabove, may describe a given set of open curves in an efficient way.According to equations (5) and (6), the derived weight function w can beused to construct a set of orthonormal polynomials where w is the weightfunction underlying the definition of a scalar product. Based on this,the corresponding Gaussian quadrature can be derived. The latter can beregarded (via calculation of the roots of specific polynomials of theaformentioned set of orthonormal functions) as an efficient resamplingstrategy for all given discrete open curves. FIGS. 6A and 6B describeone embodiment of the above described discrete curve re-samplingtechnique.

FIG. 6A—Method for Re-Sampling Discrete Curves

FIG. 6A flowcharts one embodiment of a method for efficiently sampling(re-sampling) discrete curves. As noted above, in various embodiments,some of the steps may be performed in a different order than shown, ormay be omitted. Additional steps may also be performed.

As FIG. 6A shows, in 742, each of a set of discrete curves, c₁, . . . ,c_(M), may be normalized. For example, the curves may be normalized inaccordance with the method described below with reference to FIG. 7D,although it is noted that any other normalization scheme may also beused. In one embodiment, as part of the normalization process, eachdiscrete curve may be uniformly sampled, as described below withreference to FIG. 7D. In a preferred embodiment, each (normalized)discrete curve has the same number of points.

After all of the discrete curves have been normalized, then, in 724, theweight vector w may be computed based on the discrete curves. In oneembodiment, the weight vector w may comprise a non-negative weightfunction w such that

c_(n), c_(m)

_(m)=δ_(nm) is satisfied in the least-square sense for all n and m, asnoted above in (15) and (16). In one embodiment, the weight vector maybe operable to transform discrete curves into a form which emphasized oramplifies the differences between the discrete curves.

In 746, a set of orthonormal polynomials may be calculated based on theweight vector w. As described above in the section on Sturm-LiouvilleTheory, the set of polynomials may serve as basis functions forrepresenting functions in L²([a,b],R), analogous to sine functions inFourier Theory. In particular, the mapping operator w may be regarded asa weight function defined on [−1,1] from which the set of orthonormalpolynomials may be calculated, as shown in (10). In other words, w maybe regarded as a non-negative function defined on [−1,1] and equations(6) and (1) applied to construct a set of orthonormal polynomialsrepresenting w.

$\begin{matrix}\begin{matrix}{{\sum\limits_{k = 0}^{n}\;{{p_{i}\left( x_{k} \right)}{p_{j}\left( x_{k} \right)}w_{j}}} = {\int_{- 1}^{1}{{p_{i}\left( x_{k} \right)}{p_{j}\left( x_{k} \right)}{w(x)}\ {\mathbb{d}x}}}} \\{{= {\delta_{ij}\mspace{14mu}{for}\mspace{14mu} j}},{k \leq n}}\end{matrix} & (10)\end{matrix}$

As noted above, Gaussian quadrature provides tools which translatecontinuous orthonormality relations into discrete orthonormalityrelations suitable for discrete curve operations. It is noted that theset of polynomials includes polynomials of any (or all) orders.

In 748, a polynomial may be selected from the set of polynomials of 746.In one embodiment, the polynomial may be selected such that the numberof zeros (i.e., the order) of the polynomial equals the number ofsamples or points desired in the re-sampled discrete curve. For example,if the re-sampled discrete curve is to be represented by 60 points, thena polynomial of order 60 may be selected from the set of polynomials.

In 750, the zeros of the selected polygon may be determined. In otherwords, the points are determined at which the polynomial functioncrosses the horizontal axis in the interval [−1,1]. The determined zeropoints on the interval [−1,1] indicate efficient sampling points alongthe discrete curve. In other words, the zero points provide an optimalsampling strategy for representing the discrete curve.

Finally, in 752, the discrete curves may be re-sampled according to thedetermined zero points. The zero points will tend to be clustered aroundregions of relative complexity of the discrete curve, such as curves orbends, while leaving straight, i.e., simple or uninteresting, regionssparsely sampled. It should be noted that the zero points may capturethe characteristic regions of each and all of the curves, and so for agiven curve, the sample points may include clusters in regions which aresimple, i.e., the sampling strategy which results is the same for all ofthe curves. Thus, characteristic details defining each curve may becaptured by the sampling strategy. Results of such a sampling strategyare presented below with reference to FIG. 6B. It should be noted thatthe new points can distinguish all curves under investigation, so thenumber of sampling points is reduced significantly. This results inunevenly sampled curves where essential locations are emphasized. In apreferred embodiment, the re-sampling scheme described above may be usedin curve matching applications, various embodiments of which aredescribed below with reference to FIG. 7-7D. In one embodiment, suchre-sampling may be used to reduce the data required to representdiscrete curves, i.e., may be used in a data compression scheme.

FIG. 6B—Examples of Re-Sampled Discrete Curves

FIG. 6B illustrates several examples of discrete curves re-sampledaccording to the method described above with reference to FIG. 6A. AsFIG. 6B shows, four different curves have been re-sampled. Each curveoriginally included 40 points. In this example, a polynomial of order 13was selected, with 13 corresponding zero points, resulting in 13 samplepoints for each curve. Thus, in this example, the number of pointsrequired to represent each curve was reduced by more that two thirds. Asmay be appreciated, such a reduction in the amount of informationrequired to represent a curve (without a similar reduction in accuracy)may greatly improve run-time performance of pattern matching processesbased on the curves. In one embodiment, such a sampling scheme may beused in various of the pattern matching methods described herein.

As FIG. 6B also shows, the lower panel compares the distribution ofzeros of the order 13 polynomial with the distribution for a constantpositive weight function (Gaussian quadrature).

FIG. 7—Discrete Curve Pattern Match Method

In one embodiment, a variant of equation (13) may be used to generatemapping operators W_(m) which may be used to perform curve matchingbetween a received target discrete curve and a plurality of templatediscrete curves. Is should be noted that the terms “target” and“template” are labels only, and are not intended to limit the techniquesdescribed herein to any particular types of discrete curves.

FIG. 7 is a flowchart diagram illustrating one embodiment of a methodfor locating regions of a target image that match an object of interestor a plurality of objects of interest. The object of interest maycomprise a template image, or may be included in a template image. Itshould be noted that although the method of FIG. 7 (and methods ofsubsequent Figures) is described in terms of images, the method isbroadly applicable to non-image data as well. In other words, image datais but one type of data contemplated for use in various embodiments ofthe methods described herein. It is further noted that in variousembodiments, some of the steps may occur in a different order thanshown, or may be omitted. Additional steps may also be performed.

As FIG. 7 shows, in 602, one or more mapping operators may bedetermined. The mapping operator(s) may be applicable to the templateimage(s) and optionally to one or more target images to map discretecurves representing image objects in the respective images to a form orspace which enhances or amplifies differences between the object ofinterest and objects not of interest. This step may be considered alearning stage of the process, in that the mapping operator may bedetermined based on one or both of the template image and the type oftarget images to be analyzed. For example, in one embodiment, thetemplate image (i.e., the object of interest) may be analyzed, and themapping operator determined and/or configured such that the object ofinterest may be effectively distinguished from other objects, e.g.,“background objects” or objects not of interest. In one embodiment, thetemplate image itself may include both the object of interest and one ormore background objects. The mapping operator may be configured todiscriminate effectively between the object of interest and the one ormore background objects. In another embodiment, one or morerepresentative target images which include one or more backgroundobjects may be analyzed along with the template image to configure themapping operator for effective discrimination between the object ofinterest and the background object(s). The determined mapping operatormay then be stored for use in the matching stage 604, described below.The mapping operator may be applied to objects or curves detected in thetemplate image and the resulting mapped curves stored for use in thematching stage, described below. For example, the original templatediscrete curve may characterize a first shape, and the mapped templatediscrete curve may characterize a second, different shape. Furtherdetails of the learning stage, and particularly the development of themapping operator are provided below with reference to FIG. 7A, as wellas in the Theory section which follows.

After the mapping operator has been determined, in 604 pattern matching,e.g., geometric pattern matching, e.g., curve matching, may be performedon one or more target images using the determined mapping operator tolocate one or more instances of the object of interest. This step isreferred to as the matching stage of the process. In one embodiment,after normalizing and/or re-sampling discrete curves from the targetimage(s), the curves may be compared to the mapped template discretecurves. In another embodiment, the mapping operator determined in 602may be applied to objects or curves in the target images, and patternmatching performed between these curves and the mapped curves from thetemplate image (from 602 above). Performing pattern matching on the oneor more target images may generate pattern matching results, including,for example, which, if any, of the one or more target images match theobject of interest. Further details of the matching stage are describedbelow with reference to FIG. 8.

After the pattern matching has been performed, in 606 the patternmatching results may be output, for example, to a memory store on thecomputer, to a display screen, and/or to an external system coupled tothe computer, such as a server computer system. In one embodiment, thepattern matching results may be used to trigger an action. For example,in the machine vision system illustrated in FIG. 4, the pattern matchingresults may indicate a part which does not match the template image,e.g., a part which fails a quality assurance test, and the part may berejected, as shown.

FIG. 7A—Learning Stage

FIG. 7A is a more detailed flowchart of one embodiment of the learningstage 602 of the curve matching process. As noted above, in variousembodiments, some of the steps may occur in a different order thanshown, or may be omitted. Additional steps may also be performed.

As FIG. 7A shows, in 702, the template image may be acquired. In otherwords, the target image may be received by and/or stored in the computersystem. The target image may be received from any of various sources,such as an image source coupled to the computer system over a network.

In 704, a set of discrete curves may be determined from the acquiredtemplate image. Note that as used herein, the term “discrete curve”refers to a sequence of points or vector, i.e., an ordered set ofpoints, which characterizes a curve, edge, or boundary in an image ordata set. The set of discrete curves may represent different objects inthe image, may represent different aspects of a single object, or acombination of both. Note that as used herein, the terms “curve”,“boundary”, and “edge” are used interchangeably. For example, in oneembodiment, determining the discrete set of curves from the templateimage may include performing boundary or edge detection on the templateimage to determine boundaries or edges of objects present in the targetimage. More specifically, the method may operate to detect at least oneboundary or edge present in the template image corresponding to a shapeof an object in the target image. The method may then determine at leastone discrete curve, i.e., sequence of points, from the detected boundaryor edge. It is noted that in some cases, the edges may not be visible byhuman eyesight, yet may be detectable by image analysis, e.g., bycomputer vision processes.

In one embodiment, sequences of points may comprise sequences of pixels,where each pixel has a position based on its location in the image. Itshould also be noted that in some embodiments, the points may beconsidered to be in the complex plane. In other words, the plane of theimage may be considered to be the complex plane. For example, if a pointin the image has a regular position (x,y), then its complex position is(x, iy), which may be represented by the complex number x+iy, thus eachcurve may be considered a vector of complex values. Thus, the templateimage and/or the target image is interpretable as a complex plane, wherethe sequences of points comprise sequences of pixels, and wherepositions of the pixels comprise complex values. A detailed descriptionof one embodiment of a method for determining the set of discrete curvesis presented below with reference to FIG. 7B. It is noted that the setof discrete curves may include one discrete curve, or a plurality ofdiscrete curves.

After determining the set of discrete curves, in 706 a mapping operatormay be determined based on the set of discrete curves. The mappingoperator may be operable to emphasize differences between respectiveobjects in an image. In one embodiment, the mapping operator maycomprise non-negative components. In another embodiment, the mappingoperator may comprise complex components, as described below withreference to FIG. 9B. Further details of determining the mappingoperator are described below with reference to FIG. 7C.

In 708, the mapping operator may be applied to the set of discretecurves to generate a set of mapped discrete curves for the templateimage. In other words, the mapping operator may be applied to the pointsequences which define the discrete curves of the template image,thereby generating mapped point sequences which define correspondingmapped discrete curves.

Finally, in 710, the mapping operator and the set of mapped discretecurves may be stored, for example, in a storage medium of the computersystem 102 or an external computer system coupled to the computer system102, such as a server computer. The stored mapping operator and mappedcurves may be useable to perform pattern (shape) matching on one or moretarget images, as described in 604 above.

FIG. 7B—Determining the Set of Discrete Curves

FIG. 7B flowcharts a detailed embodiment of step 704 of FIG. 7A, whereina set of discrete curves is determined from a template image. It isnoted that although the method is described as applied to the templateimage, the method also may be applied to one or more target images todetermine respective sets of discrete curves in the target images. Asmentioned above, in various embodiments, some of the steps may occur ina different order than shown, or may be omitted, and/or additional stepsmay be performed, as desired.

In 712, a curve detection threshold may be determined. The curvedetection threshold may be used to distinguish between curves, e.g.,edges or boundaries, and clusters. A cluster is a point set which doesnot qualify as a discrete curve for the purposes of pattern matching ina given application. For example, if the threshold is set too low,meaning that the curve detection is too sensitive, then noise in theimage may be detected as one or more curves which do not accuratelyreflect the object or objects to be characterized. Setting the thresholdtoo low generally results in a large number of “detected curves” whichare actually clusters. Conversely, setting the threshold too high mayresult in legitimate curves in the image going undetected. Thus, thethreshold may need to be “tuned” for a particular application toeffectively detect curves without also detecting too many clusters. Thecurve detection threshold may be based on various metrics, including oneor more of curve length, e.g., number of points in the curve (orcluster), size, such as area spanned by the points, curvature, such asstandard deviation of the curve, or any other metric useable todistinguish between curves and clusters.

Once a curve detection threshold has been determined, then in 714, edgedetection, i.e., curve detection, may be performed on the image, usingthe determined curve detection threshold to identify curves and clustersin the image. The identification of curves and clusters in the image mayresult in respective numbers of detected curves and clusters.

In 716, the number of detected curves and clusters may be analyzed, andthe threshold value adjusted accordingly. For example, if the number ofcurves detected is too great, then the threshold may be raised, or, ifthe number of curves detected is too small, the threshold value may belowered.

As FIG. 7B shows, in one embodiment, steps 714 and 716 may be repeatedone or more times to “tune” the curve detection threshold. For example,in one embodiment, the initial value of the threshold may be set to arelatively high value. The detection and analysis of steps 714 and 716may then be performed using the set value, and the results stored. Thethreshold value may then be lowered, and the detection and analysisperformed again. This process may be repeated, lowering the thresholdvalue each time and storing the results. If the detection and analysisfor a particular iteration results in a substantial increase in thenumber of detected curves, this may indicate that clusters are now beinginterpreted as curves, and so the previous threshold value (i.e., thethreshold value of the previous iteration) may be considered to be the“tuned” value. The previous detection and analysis results may then beused, if stored, or the detection and analysis may be performed againwith the tuned threshold value to regenerate the results.

Finally, the results may be output, as indicated in 718. Outputting theresults may simply mean storing the results in a particular area ofstorage for later use, or may include sending the results to anotherprogram or system.

FIG. 7C—Determining the Mapping Operator(s)

FIG. 7C flowcharts a detailed embodiment of step 706 of FIG. 7A, whereinone or more mapping operators are determined based on the set ofdetected discrete curves of the template image. As noted above, invarious embodiments, some of the steps shown may be performed indifferent order, or may be omitted, as desired. Additional steps mayalso be performed.

As FIG. 7C shows, in 722, each of the discrete curves may be normalized.A primary reason for normalizing curves is so that they may be comparedwith each other in the same terms. In various embodiments, normalizationof each curve may include any of a variety of operations, as is wellknown in the art. For example, in one embodiment, the length of thediscrete curve or sequence may be normalized to 1, or, as is commonlydone, to 2, where the relative positions of the points along the(normalized) curve length range from −1 to +1. In one embodiment, eachof the curves may also be re-sampled, e.g., over-sampled orunder-sampled, to generate normalized curves of the same number ofpoints. Thus, each of the set of discrete curves may be normalized(using the same normalization scheme, of course). In one embodiment, theoptimal re-sampling scheme described above with reference to FIG. 6A maybe used to re-sample the discrete curves.

After the curves have each been normalized, then in 724, a mappingoperator W_(m) for each curve may be determined which operates toemphasize differences between the curves. One way to do this is tocompute W_(m) for each curve c_(m) such that for each pair of curvesc_(n) and c_(m),

c_(n),c_(m)

_(w) _(m) =δ_(nm)  (15)where δ_(nm) is the Kronecker delta, is (substantially) satisfied. Inother words, if c_(n) and c_(m) are very similar, then the inner orscalar product of the two curves will be close to 1, and if the twocurves are dissimilar, the product will be close to zero. Theapplication of W_(m) to the curves may thus serve to emphasize oramplify differences between the curves. As noted above, in oneembodiment, the points which define each curve may be complex valued,and so the inner product may be calculated thus:

$\begin{matrix}\begin{matrix}{\left\langle {c_{n},c_{m}} \right\rangle_{W_{m}} = {\sum\limits_{i = 1}^{N}\;{c_{ni}c_{mi}^{*}W_{m_{i}}}}} \\{= \delta_{nm}}\end{matrix} & (16)\end{matrix}$where the product is between c_(n) and the complex conjugate of c_(m),as is well known in the art. It should be noted that the complex vectorsc_(n), c_(m), and each W_(m) preferably all have the same number ofelements. Thus, expression (16) for each pair of curves may provide aset of equations with which each W_(m) may be solved. It should also benoted that in one embodiment, a perfect solution of W_(m) (with respectto the formal definition of δ_(nm)) is not necessary, but that anapproximate solution may suffice. In other words, the solution W_(m) maynot meet the constraints of equation (16) exactly for every pair ofcurves, but may still operate to effectively discriminate betweendissimilar curves.FIG. 7D—Normalization of Discrete Curves

FIG. 7D flowcharts one embodiment of a method for normalizing one ormore discrete curves, as mentioned above in step 722 of the method ofFIG. 7C. In this example, each discrete curve is normalized based on itsmoment. It should be noted that the method shown is but one example of anormalization scheme for discrete curves, and that many othernormalization schemes may also be suitable, as is well known in the art.

In 732, each of the one or more discrete curves may optionally bere-sampled to a common sample basis. In other words, the discrete curvesmay be re-sampled such that each discrete curve has the same number ofpoints. In one embodiment, the discrete curves may be re-sampleduniformly, i.e., the sample points along each discrete curve may beevenly distributed. In another embodiment, each discrete curve may bere-sampled in accordance with the re-sampling scheme described abovewith reference to FIGS. 6A-6B. In other embodiments, the discrete curvesmay be normalized and/or re-sampled a plurality of times.

In one embodiment, each point in the discrete curve or point sequencemay be normalized based on the geometric center of the sequence ofpoints. Thus, in 734, a center or average position may be calculated foreach discrete curve. In this case, assuming N sequential points withrespective positions (x_(n)), where n=1, N, the center x_(c) may becalculated thus:

$\begin{matrix}{x_{c} = \frac{x_{1} + \ldots + x_{N}}{N}} & (17)\end{matrix}$

Each point (x_(n)) may then be normalized by subtracting the center,giving a new, normalized sequence of points:(x ₁ −x _(c)), . . . , (x _(N) −x _(c))=x ₁ ′, . . . , x _(N)′.  (18)

Then, in 738, the moment of the sequence or discrete curve may benormalized, for example, to 1. In this case, the moment of each sequenceor discrete curve may be calculated by:

$\begin{matrix}{S = {\left( {\sum\limits_{1}^{N}\;{{x_{j} - x_{c}}}^{2}} \right)^{\frac{1}{2}}.}} & (19)\end{matrix}$

The normalized point sequence is then:

$\begin{matrix}{\frac{x_{1} - x_{c}}{S},\ldots\mspace{11mu},{\frac{x_{N} - x_{c}}{S}.}} & (20)\end{matrix}$

Finally, in 740, the normalized discrete curves may be output, such asto a display, memory location, or external system, among others. Thus,each discrete curve may be normalized based on its respective moment. Asnoted above, other normalization schemes are also contemplated for usein various embodiments of the present invention.

FIGS. 8A-8B—Matching Stage

FIGS. 8A and 8B are more detailed flowchart diagrams illustratingembodiments of the matching stage described above with reference to FIG.7. More specifically, FIG. 8A illustrates a method where mappingoperators are applied to respective template discrete curves andcompared with target curves, while FIG. 8B illustrates an embodimentwhere mapping operators are applied to both template discrete curves andtarget discrete curves, and the mapped curves are then compared.

It should be noted that although the method is generally described interms of matching a single discrete curve (sequence of points), multiplediscrete curves (sequences of points) may also be detected and matchedaccording to the method described. In other words, the template imagemay include multiple discrete curves, each corresponding to a templateimage object, or to different aspects of a single object, or acombination of both. A purpose of the matching phase may be to match anyof the plurality of discrete curves in the template image to any of thediscrete curves present in one or more target images. As mentionedabove, although the methods are described in terms of images, themethods are broadly applicable to non-image data as well, i.e., imagedata is but one type of data contemplated for use in various embodimentsof the methods described herein. It is further noted that in variousembodiments, some of the steps may occur in a different order thanshown, or may be omitted. Additional steps may also be performed.

As FIG. 8A (and 8B) shows, in 802, the target image may be acquired. Inother words, the target image may be received by and/or stored in thecomputer system. The target image may be received from any of varioussources, as desired, including an image source coupled to the computersystem over a network. It is noted than in other embodiments, the targetimage may already be present in the computer system, and thus step 802may be omitted. In this embodiment, the template image is already storedin the computer system, although in other embodiments the pattern ortemplate image may be received prior to or after reception of the targetimage.

In 804, a target image discrete curve may be determined from the targetimage, corresponding to a respective object in the target image. Inother words, the target image discrete curve may correspond to one ormore geometric features of the respective object. As noted above, theterm “discrete curve” is herein used interchangeably with the term“sequence of points”. Similarly, the terms “boundary” and “edge” areused interchangeably. For example, in one embodiment, determining thetarget image discrete curve from the target image may include performingboundary or edge detection on the target image to determine boundariesor edges of objects present in the target image. More specifically, themethod may operate to detect at least one boundary or edge present inthe target image corresponding to a shape of an object in the targetimage. The method may then determine a sequence of points from thedetected boundary or edge which defines the target image discrete curve.

In one embodiment, a plurality of target image discrete curves may bedetermined from the target image. Each of the plurality of target imagediscrete curves may correspond to a respective image object or to arespective portion of an image object in the target image. Said anotherway, there may be multiple objects (or aspects of a single object) inthe target image, each represented by a respective discrete curve.

As FIG. 8A indicates, after the determination of the target imagediscrete curve(s), in 808A, curve matching may be performed on thetarget image discrete curve(s) and one or more mapped template imagediscrete curves, where each of the mapped template image discrete curvecorresponds to an object of interest, or to a portion of the object ofinterest. In performing the curve matching a distance may be computedfor the target image discrete curve relative to the mapped templateimage discrete curve. The computed distance may be a metric whichindicates a degree of difference between the target image discrete curveand each of the one or more mapped template image discrete curves, andthus, may indicate a degree of difference between the object in thetarget image and the object of interest. As described above in theTheory section, in one embodiment, the distance metric may comprise ascalar product, i.e., an inner or dot product, between the mapped orweighted template discrete curve and the target discrete curve, wherevalues close to unity indicate a substantial match.

In an alternate embodiment, illustrated in 806 of FIG. 8B, the mappingoperator(s) may be applied in a different manner. Rather than applyingthe mapping operator(s) W_(m) only to the template discrete curves, inone embodiment, the mapping operator may be applied to both the templatediscrete curve and the target discrete curve(s). For example, themapping operator may be the square root of W_(m)., i.e., W_(m) ^(1/2) Inthis case, both the template discrete curve and the target discretecurve may be mapped respectively to a mapped template discrete curve anda mapped target discrete curve, and the two mapped curves compared. Itshould be noted that this alternate approach is mathematicallyequivalent to mapping only the template discrete curve using W_(m).However, in this alternate embodiment, the symmetric application of themapping operator (W_(m) ^(1/2)) may be interpreted as mapping bothdiscrete curves into forms which enhance differences between the curves.For example, the mapping operator may be applied to the templatediscrete curve in the learning stage of the process, as described abovewith reference to FIG. 7A, then, in the matching stage, the mappingoperator may be applied to the target discrete curve, as indicated in806 of FIG. 8B.

Thus, in this alternate embodiment, after the target image discretecurve (or target image discrete curves) has been determined, then in806, the target image discrete curve may be mapped to a different mappedtarget image discrete curve using a mapping operator. The mappingoperator, W(x), may operate to transform the target image discrete curveinto a form (the mapped target image discrete curve) which is moreeasily distinguishable from (mapped) discrete curves corresponding toobjects not of interest in the target image. In other words, the mappingoperator may operate to emphasize differences between respectiveobjects, or aspects of objects. In the descriptions that follow,references to target discrete curves and comparisons between the targetdiscrete curves and the mapped template discrete curves may beinterpreted in terms of the method of FIG. 8A or the method of FIG. 8B.In other words, in embodiments corresponding to the method of FIG. 8B,the target discrete curves may comprise mapped target discrete curves.After the target discrete curve (or curves) has been mapped to a mappedtarget discrete curve, then in 808B, curve matching may be performed onthe mapped target image discrete curve(s) and one or more mappedtemplate image discrete curves, as indicated in FIG. 8B.

In an embodiment where the target image includes a plurality of targetimage discrete curves, curve matching may be performed on each of thetarget image discrete curves relative to the mapped template imagediscrete curve. In this case a distance measure may be computed for eachof the target image discrete curve relative to the mapped template imagediscrete curve. In the alternate embodiment described above, each of thetarget discrete curves may have been mapped to a corresponding pluralityof mapped target image discrete curves prior to, or as part of, thecomputation of the distance measure.

It should be noted that in some embodiments, the template image mayinclude multiple discrete curves, i.e., object of interest sequences ofpoints, or there may be multiple template images from which the templatediscrete curves may be determined. In other words, there may be aplurality of objects of interest, or, the object of interest may includea plurality of sub-objects, each represented by a respective discretecurve. In this case, curve matching may be performed on each of thetarget image discrete curves relative to each of the mapped templateimage discrete curves, computing a distance measure for each of thetarget image discrete curves relative to each of the mapped templateimage discrete curves.

In one embodiment, the mapped template image discrete curve may begenerated from a characterization of the object of interest. In otherwords, the method may include characterizing the object of interest toproduce the mapped template image discrete curve, for example, in thelearning stage, as described above with reference to FIG. 7A. Thus,characterizing the object of interest may be performed prior toacquiring the target image. Characterizing the object of interest mayinclude determining a template image discrete curve from informationregarding the object of interest, where the template image discretecurve corresponds to the object of interest. The template image discretecurve may then be mapped to the mapped template image discrete curveusing the mapping operator. In a preferred embodiment, the informationregarding the object of interest comprises a template image containingthe object of interest, i.e., the template image referred to above. Ofcourse, as mentioned above, in other embodiments, the discrete curvesmay be determined from non-image data sets rather than images.

It is noted that because the mapping operator emphasizes differencesbetween curves, i.e., sequences of points, the distance between thetarget image discrete curve (or the mapped target image discrete curve)and the mapped template image discrete curve may be greater than adistance computed between the target image discrete curve and thetemplate image discrete curve. In other words, the distance computedbetween the target image discrete curve and the template image discretecurve indicates a degree of difference between the original detecteddiscrete curve in the target image (the target image discrete curve) andthe original detected discrete curve in the template image (the templateimage discrete curve, i.e., the object of interest), while the distancebetween the (mapped) target image discrete curve and the mapped templateimage discrete curve indicates a degree of difference between the(mapped) discrete curve in the target image and the mapped discretecurve from the template image (the mapped template image discretecurve). Because the mapping operator serves to amplify the differencesbetween the original curves, the mapped curve(s) reflect a greaterdistance measure.

In one embodiment, the mapping operator may be a discrete curve mappingoperator which maps a first discrete curve to a second discrete curve.As mentioned above, the term “discrete curve” refers to an ordered setor sequence of points which have a certain direction. Furthermore,concatenated versions of these points do not generate intersections. Inother words, the curve is simple, although in various embodiment, thecurve may be open or closed. In various embodiments, the mappingoperator may be a linear mapping operator, and/or a non-linear mappingoperator. In one embodiment, the mapping operator may be aSturm-Liouville operator. For more information on Sturm-Liouvilleoperators, please see the Theory section above.

In one embodiment, the mapped discrete curve (either the mapped templatediscrete curve or the mapped target discrete curve) may be a uniquediscrete curve generated from the template or target image discretecurve. In other words, for any template or target image discrete curve,the mapping operator may be operable to generate a second, uniquediscrete curve. The mapped image discrete curve may be more stable toimage disturbances than the original image discrete curve. Said anotherway, the mapped image discrete curve may be more robust or lesssusceptible to disturbances, e.g., noise, distortions, or perturbations,than the original image discrete curve.

It is noted that in one embodiment, the boundary (or boundaries)detected in the target image may correspond to or define at least onevisual boundary that is visually identifiable in the target image, i.e.,the boundary may be a visibly distinguishable geometric feature of anobject in the target image. In contrast, the mapped target imagediscrete curve may not define a visual boundary that is visuallyidentifiable in the target image. In other words, the mapping operatormay transform the boundary or edge defined by the target image discretecurve into a curve (defined by the mapped target image discrete curve)which bears little resemblance to the original curve (e.g., the originaledge or boundary).

Similarly, whereas the boundary (or boundaries) detected to be presentin the target image may correspond to or define at least one shape thatis visually identifiable in the object, the mapped target image discretecurve may not define a shape that is visually identifiable in theobject. Said another way, the target image discrete curve may define atleast one shape or visual boundary that is visually identifiable in thetarget image, while the mapped target image discrete curve may define ashape or visual boundary that is not visually identifiable in the targetimage.

In embodiments where non-image data are analyzed, each discrete curvemay correspond to a respective data object, or to a portion of a dataobject. In other words, each discrete curve may define a boundary, edge,or shape in the data.

In one embodiment mapping the template or target image discrete curve tothe different mapped template or target image discrete curve may includeapplying the mapping operator to each point in the template or targetimage discrete curve to generate corresponding points in the mappedtemplate or target image discrete curve. For example, the target imagediscrete curve may be considered to be a first vector, where the mappingoperator W and the first discrete curve vector have the same number ofelements. In this embodiment, mapping the target image discrete curve tothe different mapped target image discrete curve may include multiplyingeach element of the mapping operator W with corresponding elements inthe first vector to generate a second vector. The elements of the seconddiscrete curve vector may then comprise the mapped target image discretecurve. This description applies to the template image discrete curve(s),as well.

As also mentioned above, in some embodiments, the elements of the firstdiscrete curve vector may be complex. In this case, multiplying eachelement of the mapping operator W with corresponding elements in thefirst discrete curve vector may comprise multiplying each element of themapping operator W with complex conjugates of corresponding elements inthe first discrete curve vector, as is well-known in the art.

In 808A of FIG. 8A, curve matching may be performed on the target imagediscrete curve and a mapped template image discrete curve, where themapped template image discrete curve corresponds to the object ofinterest. In performing the curve matching a distance may be computedfor the target image discrete curve relative to the mapped templateimage discrete curve. The computed distance may be a metric whichindicates a degree of difference between the target image discrete curveand the mapped template image discrete curve, and thus, may indicate adegree of difference between the object in the target image and theobject of interest. In embodiments where there are multiple templatediscrete curves, then the curve matching may be performed between thetarget discrete curve and each of the template discrete curves.

Alternately, in the embodiment of FIG. 8B, after the target imagediscrete curve has been mapped in 806, then in 808B, curve matching maybe performed on the mapped target image discrete curve and the mappedtemplate image discrete curve, where the mapped template image discretecurve corresponds to the object of interest, resulting in a computeddistance for the mapped target image discrete curve relative to themapped template image discrete curve.

Finally, in 810, pattern matching results may be generated based on thedistance or distances computed in 808 (A or B). In one embodiment, thecomputed distance may be compared to a threshold value, and if thedistance is less than the threshold value, the object in the targetimage represented by the (mapped) target image discrete curve may beconsidered a match with respect to the object of interest represented bythe mapped template image discrete curve. It is noted that a computeddistance value of zero indicates a perfect match. In one embodiment, amatching score for the target image object may be calculated based onthe proximity of the distance value to zero. Thus, generating patternmatching results may comprise generating information indicating one ormore matches for the object of interest in the target image.

In an embodiment where the target image includes a plurality of targetimage discrete curves, each of which may be optionally mapped to acorresponding plurality of mapped target image discrete curves, patternmatching results may be generated based on the distance measures foreach of the (optionally mapped) target image discrete curves withrespect to the mapped template image discrete curves. Similarly, whenthe template image includes a plurality of discrete curves, patternmatching results may be generated based on the distance measures foreach of the (optionally mapped) target image discrete curves withrespect to each of the mapped template image discrete curves.

In one embodiment, after the pattern matching results have beengenerated in 810, the method may include outputting the pattern matchingresults. In various embodiments, outputting the pattern matching resultsmay include, for example, sending the results to an external system,such as to an external computer over a network, displaying the resultson a display, such as a computer display or a printer, storing theresults in a particular memory location or log, or signaling a system orprogram to perform a task.

In one embodiment, the mapping operator(s) may be re-calculated to moreeffectively match and/or discriminate between template image discretecurves and target image discrete curves, e.g., using information relatedto one or more target images. For example, a new or modified mappingoperator W (or a plurality of new or modified mapping operators W_(m))may be computed using the template image discrete curves and discretecurves from additional target images or discrete curves which may nothave been included in the first calculation of W. In variousembodiments, the original mapping operator(s) W may be replaced, or maysimply be modified using the additional discrete curve information.Thus, some or all of the terms of the mapping operator(s) may beconfigurable for different objects of interest. In other words, themapping operator may be computed or adjusted to “tune” the operator forparticular objects or sets of objects.

In one embodiment, the mapping operator may include at least oneparameter that is configurable to enhance differences between respectiveobjects of interest and objects not of interest. In other words, themapping operator may include a parameter which may be adjusted to “tune”the operator for particular objects or sets of objects. Said anotherway, given an object of interest and one or more objects not ofinterest, the at least one parameter may be adjustable to particularlyemphasize differences between that object of interest and the one ormore objects not of interest.

For example, in one embodiment, first information may be receivedregarding the object of interest, e.g., which characterizes the templateimage. Second information regarding one or more objects that are not ofinterest may also be received. At least one parameter for the mappingoperator may then be determined based on the first and secondinformation, where the parameter may be determined to enhance a distancebetween mapped discrete curves or point sequences corresponding to theobject of interest and mapped discrete curves or point sequencescorresponding to the one or more objects that are not of interest. Themapping operator may then be configured according to the determinedparameter(s).

Thus, the mapping operator may be computed, adjusted or tuned to enhanceor increase a distance between mapped discrete curves (point sequences)corresponding to the object of interest and mapped discrete curves(point sequences) corresponding to objects that are not of interest.

Shift, Scale, and Rotation Invariance

A more difficult situation arises when rotations of all discrete curvesare also valid transforms. Conceptually, formulas (13) and (14) canstill be used to perform the matching procedure. However, in thisapproach, instead of looking for numbers in formula (1) that are closeto 1, the goal is to look for a complex number that has a magnitudeclose to 1. The phase of the complex number represents the relativeangle between the original curve and the match found.

Let c₁, . . . , c_(M) be complex discrete curves of same size in thecomplex plane. The best match to a curve c must be determined.

FIGS. 9A and 9B—Matching of Discrete Open Curves with Shift, Scale, andRotation Invariance

FIGS. 9A and 9B illustrate further embodiments of the method describedabove with reference to FIG. 7. More specifically, FIG. 9A flowchartsone embodiment of a method for matching discrete open curves with shift,scale, and rotation invariance, where a received image (or data set) issearched for the presence of an object of interest using a non-negativemapping operator (or a plurality of non-negative mapping operators). Theobject of interest may comprise a template image, or may be included ina template image. It is noted that in various embodiments, some of thesteps may occur in a different order than shown, or may be omitted.Additional steps may also be performed.

As FIG. 9A shows, in 722 a set S of template image discrete curves, c₁,. . . , c_(M), may be normalized, as described above with reference toFIGS. 7C and 7D. For example, the curves may be normalized in accordancewith the method described above with reference to FIG. 7D, although itis noted that any other normalization scheme may also be used. In oneembodiment, as part of the normalization process, each discrete curvemay be uniformly sampled, as described above in step 732 of FIG. 7D. Ina preferred embodiment, each (normalized) discrete curve has the samenumber of points.

After all of the discrete curves have been normalized, then, in 904,mapping operators W_(m) may be computed based on the discrete curves, asdescribed above with reference to FIG. 7C. In one embodiment, eachmapping operator W_(m) may comprise a non-negative mapping functionW_(m) such that

c_(n),c_(m)

_(w) _(m) =δ_(nm) is satisfied in the least-square sense for all n andm, as noted above in (15) and (16). Said another way, M non-negativemapping functions W₁, . . . , W_(M) may be computed such that

c_(n),c_(m)

_(w) _(m) =δ_(nm) is (substantially) satisfied (usually, in theleast-square sense) for all n and m. In one embodiment, the constraintof smoothly changing components of W_(m) may also be added. As alsodescribed above, the mapping operators may be operable to map discretecurves into a form which emphasizes or amplifies the differences betweenthe discrete curves.

In 908, degenerate (rotated) discrete curves may optionally be removedfrom the template image curves. In other words, if one (or more) of thenumbers

c_(n),c_(m)

_(w) _(m) for n≠m has (have) a magnitude of close to 1, then the curvesare not independent. In other words, they are rotated versions of othercurves in S, thus, it may be necessary to eliminate such curves from S.

It may be noted that steps 722, 904, and 908 above together may composeone embodiment of a learning stage 602A, as referred above withreference to FIG. 7.

Once the degenerate curves have (optionally) been removed from the setof discrete curves S, then in 910, a curve c from a target image (ordata set) may be acquired and normalized and/or re-sampled. In otherwords, curve c may be acquired from the target image, then normalizedand/or re-sampled such that the discrete curve c has the same number ofpoints as each of the template curves. Additionally, in one embodiment,the curve c may be re-sampled in accordance with a sampling scheme usedfor each of the template curves.

Then, in 912, for each template image curve c_(m), the complex value

c,c_(m)

_(w) _(m) may be computed. Once all of the complex values for

c,c_(m)

_(w) _(m) have been calculated, then in 914, the maximum value may bedetermined, where the maximum value corresponds to a best match amongthe template image discrete curves. In other words, the magnitudes ofall of the computed complex numbers may be determined, and the maximummagnitude ascertained. The index m belonging to this maximum representsthe best match among the given set c₁, . . . , c_(M). The phase of thiscomplex number may be used to determine the angle between c and c_(m).

As noted above, in one embodiment, the mapping operator may beconsidered to be W_(m) ^(1/2), in which case the mapping operator may beapplied to both the template image curve c_(m) and the target curve c.

Finally, in 606, the output pattern matching results may be output,e.g., for use by the system, or by an external system, as desired. It isnoted that steps 910, 912, and 914 above together compose one embodimentof a matching stage 604A, as referred above with reference to FIG. 7.

Non-positive Weights

The original non-negativity constraint of the weight function W followsdirectly from the identity:

$\begin{matrix}\begin{matrix}{{\sum\limits_{n = 1}^{N}\;{\left( {y_{n + j}v_{n}} \right)\left( {z_{n}v_{n}} \right)^{*}}} = {\sum\limits_{n = 1}^{N}\;{y_{n + j}z_{n}^{*}v_{n}v_{n}^{*}}}} \\{= {\sum\limits_{n = 1}^{N}\;{y_{n + j}z_{n}^{*}w_{n}}}}\end{matrix} & (21)\end{matrix}$where v is a complex weight function, and y and z are discrete curves inthe plane. The constraint may be expressed thusly:w_(n)=v_(n)v_(n)*=|v_(n)|². The quality of matching routines may befurther improved when formula (21) is replaced with the more generalversion:

$\begin{matrix}{\left\langle {y,z} \right\rangle_{w} = {\sum\limits_{n = 1}^{N}\;{y_{n + j}z_{n}^{*}w_{n}}}} & (22)\end{matrix}$where w is a complex discrete weight function that can vary with z.Compared to (21) the new formula (22) may offer many new degrees offreedom.

FIG. 9B is a flowchart of yet another embodiment of the method describedabove with reference to FIG. 7. More specifically, FIG. 9B flowcharts anembodiment of the method presented in FIG. 9A, but where the weightfunction is complex as opposed to non-negative. As before, let c₁, . . ., c_(M) be given complex discrete curves of same size in the complexplane. The best match to a curve c may then be determined. As FIG. 9Bshows, all of the steps other than 905 are the same as those in FIG. 9A,and so the descriptions are abbreviated.

As FIG. 9B shows, in 722 a set S of template image discrete curves, c₁,. . . , c_(M), may be normalized, as described above. In a preferredembodiment, each (normalized) discrete curve has the same number ofpoints.

After all of the discrete curves have been normalized, then, in 905,complex mapping operators W_(m) may be computed based on the discretecurves, as described above with reference to FIG. 7C. In one embodiment,each mapping operator W_(m) may comprise a complex weight function W_(m)such that

c_(n),c_(m)

_(w) _(m) =δ_(nm) is satisfied in the least-square sense for all n andm, as noted above in (15) and (16). Said another way, M complex weightfunctions W_(m) may be computed such that

c_(n),c_(m)

_(w) _(m) =δ_(nm) is (substantially) satisfied (usually, in theleast-square sense) for all n and m. As mentioned above, in oneembodiment, the constraint of smoothly changing components of W_(m) mayalso be added.

In 908, degenerate (rotated) discrete curves may optionally be removedfrom the template image curves, and in 910, a curve c from a targetimage may be acquired and normalized. Then, in 912, for each templateimage curve c_(m), the complex value

c,c_(m)

_(w) _(m) may be computed. Once all of the complex values for

c,c_(m)

_(w) _(m) have been calculated, then in 914, the maximum value may bedetermined, where the maximum value corresponds to a best match amongthe template image discrete curves. The curve from the set whichproduces the maximum magnitude represents the best match among the givenset c₁, . . . , c_(M). As noted above, the phase of this complex numbermay be used to determine the angle between c and c_(m).

Finally, in 606, the output pattern matching results may be output,e.g., for use by the system, or by an external system, as desired.

The method of FIG. 9B can also be regarded as a mechanism translatingthe original set of curves into a second set guaranteeing both(substantial) orthonormality of curves and the property to be a discretecurve (slowly varying components of w).

FIG. 9C demonstrates the application and results of the transform, asdescribed above with reference to FIG. 9B, applied to a specific set ofdiscrete open curves, and also compares the magnitudes of thecross-correlation of the original curves with that of the transformedcurves. As FIG. 9C shows, the left column of curves is the original setof discrete open curves, and the right column of curves is thetransformed set. In other words, the discrete open curves in the leftcolumn are transformed (by application of the mapping operator w) togive the curves in the right column. As may been seen, thetransformation may amplify or enhance characteristic curvatures of eachcurve, thereby increasing the ability of the method to discriminate andmatch between curves.

FIG. 9C also shows cross-correlation for the original curves (topfigure), and for the transformed curves (bottom figure). In an idealcase in which each curve is orthonormal to every other curve, thecross-correlation chart would simply show data points with value 1forming a diagonal, with all other values equal to zero. In other words,correlation of a particular curve with itself gives a correlation of 1,while correlation of the curve with any other curve results in acorrelation of zero. Of course, in reality, the results are not so clearcut. However, as FIG. 9C shows, the cross-correlation of theuntransformed curves is clearly less efficient than thecross-correlation of the transformed curves. Said another way, thecorrelation of the transformed curves is closer to the ideal case, inthat after the transform has been applied, the curves are much closer tomutual orthogonality than the original curves.

Optimal Discrete Closed Curve Descriptors in 2D

The above methods provide means for matching discrete open curves.However, there are many cases where the curves to be analyzed andmatched are closed curves. Closed curves do not have a unique start andend, and so a matching process must take into account the fact thatrotations are possible. As described below, a function may be definedwhich provides a similarity measure between closed curves.

Let y=(y₁, . . . , y_(N)) and z=(z₁, . . . , z_(N)) be two discreteclosed curves in the complex plane, C. Let y and z be normalized, i.e.

${\sum\limits_{n = 1}^{N}\; y_{n}} = {{\sum\limits_{n = 1}^{N}\; z_{n}} = {{0\mspace{14mu}{and}\mspace{14mu}{\sum\limits_{n = 1}^{N}\;{y_{n}}^{2}}} = {{\sum\limits_{n = 1}^{N}\;{z_{n}}^{2}} = 1}}}$

According to the first part of (5), the distance between these twocurves y and z is

$\begin{matrix}{{d_{full}\left( {y,z} \right)} = \sqrt{1 - {{\sum\limits_{n = 1}^{N}\;{y_{n}z_{n}^{*}}}}^{2}}} & (23)\end{matrix}$

Formula (23) can be used to define a similarity measure between y and zthat is constant when y and/or z are shifted cyclically and/or arerotated in reverse order.

FIG. 10A—Discrete Closed Curves

FIG. 10A illustrates two examples of discrete closed curves, defined bypoint sequences I, II, III, IV, and 1, 2, 3, 4, respectively, followinga counter-clockwise numbering convention. In this example, themagnitudes of all cross-correlation and convolution components varybetween 0.2018 and 0.9213. The best match of 0.9213 is achieved when (I,II, III, IV) is directly compared with (3,4,1,2). Further examples ofdiscrete closed curves are described below.

The above equation (23) provides a basis for the following theorem:

Theorem 1:

Let

${d\left( {y,z} \right)} = {1 - {\max\limits_{\underset{{j = 0},\ldots,{N - 1}}{{s = 0},1}}{{{\sum\limits_{n = 1}^{N}\;{y_{{{({- 1})}^{2}n} + j}z_{n}^{*}}}}^{2}.}}}$d is a metric in the space f representing all normalized discrete closedcurves of size N in the complex plane. The derived distance in the spaceof arbitrary discrete closed curves in the complex plane (d applied tonormalized curves) is a semi-metric.

According to Theorem 1, two discrete closed curves in the complex planecan be regarded as similar if and only if the cyclic cross-correlationor cyclic convolution of normalized versions of y and z generate amaximal magnitude of almost 1. In what follows it is assumed that thereis a natural order of all points in y and z in the complex plane, i.e.there is no need to take cyclic convolutions into account.

This definition of distances in spaces of discrete closed curves worksvery well for all mathematically defined curves. However, in manyapplication fields one has to deal with noisy data and more robustmethods are desirable that are based on additional information regardingthe structure of the underlying discrete curves. Optical characterrecognition and geometrically oriented pattern matching are twoexamples, among many others.

For that reason discrete closed curves without symmetry propertiesshould generate cross-correlations with a pronounced peak when the shiftis 0. Unfortunately, peaks are rarely extremely sharp for typicalcurves. For example, the cross-correlation of the character “D” in FIG.4 is almost flat. The latter result is undesirable because ofambiguities regarding rotated versions of this curve. The goal is aclear distinction between all these rotated versions.

More precisely, let be given a discrete closed curve d=(d₁, . . . ,d_(N)) of size N in the complex plane. Let W be an unknown complexweight function. Ideally, W is a smooth discrete closed curve itself andsatisfies equations

$\begin{matrix}\begin{matrix}{\left\langle {d^{k},d} \right\rangle_{W} = {\sum\limits_{n = 1}^{N}\;{d_{n + k}d_{n}^{*}W_{n}}}} \\{{= {{\delta_{k\; 0}{\mspace{11mu}\;}{for}\mspace{14mu} k} = 1}},\ldots\mspace{11mu},N}\end{matrix} & (24)\end{matrix}$

In practical situations this goal is not attainable. However, the weakersystem

$\begin{matrix}{{{{\sum\limits_{n = 1}^{N}{d_{n + k}d_{n}^{*}W_{n}}} = {{\delta_{k\; 0}\mspace{14mu}{for}\mspace{14mu} k} = 1}},\ldots\mspace{11mu},N}{{{W_{k + 1} - W_{k}} = {{0\mspace{14mu}{for}\mspace{14mu} k} = 1}},{\ldots\mspace{11mu} N}}} & (25)\end{matrix}$where W_(N+1)=W₁ can be solved in the least-square sense. The new curvedW* can be normalized. The second expression expresses the idea that Wshould be “continuous”, i.e. that differences between neighboring termsof W should be relatively small. FIG. 10C, described below, depicts theresult of such an optimization routine. As desired, thecross-correlation between d and dW* avoids ambiguities regarding peaks.

A robust matching routine can be derived based on (25), as describedbelow with reference to FIG. 10B. Let d be a given discrete and closedcurve in the complex plane that is known in advance and let c be asecond curve of same size that is acquired during runtime.

FIG. 10B—Matching Against A Given Discrete Closed Curve

FIG. 10B flowcharts one embodiment of a method for matching a discreteclosed curve c against rotated versions of a discrete closed templatecurve d. As mentioned previously, in various embodiments, some of thesteps shown may be performed in different order, or may be omitted, asdesired. Additional steps may also be performed.

As FIG. 10B shows, in 1002, a discrete closed curve d may be normalized,as described above. Then, in 1004, a complex mapping operator W may becomputed such that equations (24) and (25) hold.

In 1008, the curve dW* may be normalized. In other words, once thecomplex mapping operator W is computed in 1004, the complex conjugate ofW may be applied to the curve d and the resulting curve normalized.

In 1010, a target discrete closed curve c may be acquired andnormalized, e.g., as described above. Then, in 1012, for each cyclicshift of the curve d, the complex number

c,d^(k)

_(W) may be calculated, i.e.,

c,d^(k)

_(W) for k=0, . . . , N−1 It should be noted that cyclically permuting crather than d gives the same results, and is equivalent.

Then, in 1014, the maximum of the complex values computed in 1012 may bedetermined and compared to unity to determine the best match among thetemplate image discrete curve cyclic shifts d^(k). The comparison of thebest match to unity is useful in that in general, the match isacceptable if and only if the corresponding magnitude is close to 1 invalue. The particular cyclic shift k of the best match indicates theangle of rotation.

Finally, in 606, the pattern matching results, e.g., the best match, maybe output, as described above.

Thus, the method “orthogonalizes” the given closed curve. In other wordsthe original curve is as “orthogonal” to all rotated versions aspossible. This enables the method to decide (a) whether two closedcurves match and (b) determine the angle of rotation reliably. Thus, anembodiment of the method of FIG. 9B may be applied to discrete closedcurves, where the plurality of discrete curves c₁, . . . , c_(M)comprise rotated versions of the discrete curve d.

FIG. 10C—Example of Closed Curve Matching

FIG. 1C illustrates an example of discrete closed curve matching, asperformed by the method of FIG. 10B, described above. In FIG. 10C, thecharacter “D” (left) is translated into a new closed curve (right) wherethe correlation between this curve and all cyclically shifted versionsof the original curve shows a pronounced peak when the shift is zero.The curves are left open intentionally to depict the one-to-onecorrespondence between them.

One can even give up the requirement that W is a weight function, i.e.the optimization problem can be reformulated as

$\begin{matrix}{{{{\sum\limits_{n = 1}^{N}{d_{n + k}d_{n}^{*}W_{n}}} = {{\delta_{k\; 0}\mspace{14mu}{for}\mspace{14mu} k} = 1}},\ldots\mspace{11mu},N}{{{W_{k + 1} - W_{k}} = {{0\mspace{14mu}{for}\mspace{14mu} k} = 1}},{\ldots\mspace{11mu} N}}{{{where}\mspace{14mu} W_{N + 1}} = W_{1}}} & (26)\end{matrix}$

The approach (26), though more general, may be numerically less robustthan (25). Thus, in preferred embodiments of the present invention, theuse of (25) is preferred to that of (26).

The generalization of the method of FIG. 10B to the case of multiplediscrete closed curves may be made as follows. As before, closed curvesc_(i) (i=1, . . . , M) can be regarded as point sets in the complexplane. Let c_(i) be normalized, i.e. the mean value equals zero, and thesum of lengths of all complex numbers in c_(i) equals to 1. Let

$\begin{matrix}{\left\langle {c_{n}^{i},c_{m}^{j}} \right\rangle_{W} = {\sum\limits_{k = 1}^{N}{c_{n,{i + k}}c_{m,{j + k}}^{*}W_{k}}}} & (27)\end{matrix}$denote the weighted scalar product of closed curves c_(n) and c_(m),where the former is cyclically shifted by i and the latter by j. Theunknowns W₁, . . . , W_(N) are non-negative real numbers (composing themapping operator). To improve the quality of the matching procedure, itis desirable that the following identities hold true.

c_(n) ^(i),c_(m) ^(j)

_(W)=δ_(n+i,m+j)  (28)

In practical applications (28) cannot be achieved exactly. Equation (28)may be replaced with the following weaker optimization problem.

$\begin{matrix}{{{\underset{{j = 1},\ldots\mspace{11mu},N}{\sum\limits_{n \neq m}}{\left\langle {c_{n}^{0},c_{n}^{0}} \right\rangle_{W}}^{2}} = {\min!}}{{\left\langle {c_{n}^{0},c_{n}^{0}} \right\rangle_{w} = {{1\mspace{14mu}{for}\mspace{14mu} n} = 1}},\ldots\mspace{11mu},M}{W_{1},\ldots\mspace{11mu},{W_{N} \geq 0}}} & (29)\end{matrix}$

It can be shown that (29) results in an optimization problem of the formW^(T)CW=min!AW=bW₁, . . . , W_(N)≧0  (30)where the matrix C is square, symmetric, and positive definite. It maybe shown that b=(1, . . . , 1)^(T), A is a square real matrix withpositive entries, where the rows are normalized in the sum-of-squaressense. It follows that W=(1, . . . , 1)^(T) is a first feasiblesolution. Again, efficients algorithms for (30) exist, e.g. Active SetMethod or Hildreth-d'Esopo Algorithm (e.g. Bronshtein and Semendyayev[1997]).FIG. 11—Matching Against a Set of Discrete Closed Curves

FIG. 11 is a flowchart of one embodiment of a method for matching atarget curve against a set of discrete closed curves. The method of FIG.11 is similar to that of FIG. 10B, but matches against a plurality oftemplate discrete closed curves (and their cyclic shifts), as opposed toa single template curve (and its cyclic shifts). As mentionedpreviously, in various embodiments, some of the steps shown may beperformed in different order, or may be omitted, as desired. Additionalsteps may also be performed.

As FIG. 11 shows, in 1102, each of a set of discrete closed curves d₁, .. . d_(M) may be normalized, as described above.

Then, in 1104, using the normalized set of discrete closed curves, anon-negative mapping operator w may be computed such that equations (29)and (30) hold. In other words, w satisfies:

$\begin{matrix}{{{\underset{{j = 1},\ldots\mspace{11mu},N}{\sum\limits_{n \neq m}}{\left\langle {d_{n}^{0},d_{n}^{0}} \right\rangle_{W}}^{2}} = {\min!}}{{\left\langle {d_{n}^{0},d_{n}^{0}} \right\rangle_{W} = {{1\mspace{14mu}{for}\mspace{14mu} n} = 1}},\ldots\mspace{11mu},M}{W_{1},\ldots\mspace{11mu},{W_{N} \geq 0}}{and}} & (29) \\{{{W^{T}{DW}} = {\min!}}{{AW} = b}{W_{1},\ldots\mspace{11mu},{W_{N} \geq 0}}} & (30)\end{matrix}$

In 1010, a target discrete closed curve c may be acquired andnormalized, e.g., as described above.

Then, in 1012, for each template discrete closed curve d, the complexnumber

c,d^(k)

_(W) may be calculated for each cyclic shift of the respective curve d,i.e.,

c,d_(m) ^(k)

_(W) for k=0, . . . , N−1, m=1, . . . M. As noted above, an equivalentcalculation is

c^(k),d_(m)

_(W) for k=0, . . . , N−1, m=1, . . . M, where curve c is cyclicallyshifted, rather than d_(m).

Then, in 1014, the maximum of the complex values computed in 1012 may bedetermined and compared to unity to determine the best match among thetemplate image discrete curve cyclic shifts d^(k) for each curve d_(m).The match is acceptable if and only if the corresponding magnitude isclose to 1 in value. The particular cyclic shift k of the best matchcurve indicates the angle of rotation between curve c and the optimalmatch d^(k) _(m).

Finally, in 606, the pattern matching results, e.g., the best match, maybe output, as described above.

Closed Curve Descriptors In 3D

In some applications, it may be desirable to analyze and match higherdimensional curves, for example, 3-dimensional curves. The followingdescribes an approach for matching rotated versions of a 3D curve.

Let y=(y₁, . . . , y_(N)) and z=(z₁, . . . , z_(N)) be two discreteclosed curves in R³, i.e., in real-valued 3D space. Let y and z benormalized, i.e.

${\sum\limits_{n = 1}^{N}y_{n}} = {{\sum\limits_{n = 1}^{N}z_{n}} = {{0\mspace{14mu}{and}\mspace{14mu}{\sum\limits_{n = 1}^{N}{y_{n}}^{2}}} = {{\sum\limits_{n = 1}^{N}{z_{n}}^{2}} = 1}}}$

Let the 3D curve z be a rotated version of y. For mathematical reasons,the rotation angle between the two curves is required to be somewhatlarger than zero, i.e., the algorithm is not applicable for smallrotations. However, this is not problematic because one can easily checkwhether y and z are identical beforehand.

A fast method for generating the underlying rotation is desired. To thisend let q be the axis of rotation and T be the angle of rotation aboutthis axis q. The vector q is perpendicular to all vectors y_(n)−z_(n).Moreover, q is parallel to all of the vectors(y_(n+1)−z_(n+1))×(y_(n)−z_(n)), i.e. (31) provides a good estimate forthe axis of rotation q.

$\begin{matrix}{q = \frac{\sum\limits_{n = 1}^{N}{\left( {y_{n + 1} - z_{n + 1}} \right) \times \left( {y_{n} - z_{n}} \right)}}{{\sum\limits_{n = 1}^{N}{\left( {y_{n + 1} - z_{n + 1}} \right) \times \left( {y_{n} - z_{n}} \right)}}}} & (31)\end{matrix}$

Assume q=(0,0,1). In a second step it will be shown how to reduce thecase of a general vector q to this specific situation.

In case of q=(0,0,1) the matching process in 3D can be translated intotwo separate routines in the complex plane C, and on the real numberline, R. To this end, a 3D vector may be split into components asfollows:(y_(n,1),y_(n,2),y_(n,3))

(_(y,1)+iy_(n,2)) and (y_(n,3))(z_(n,1),z_(n,2),z_(n,3))

(z_(n,1)+iz_(n,2)) and (z_(n,3))

The center of each of these objects is 0, but they are not normalized.Using the matching strategies in C and R it follows that:

$\begin{matrix}{{{{{\sum\limits_{n = 1}^{N}{\left( {y_{n,1} + {{\mathbb{i}}\; y_{n,2}}} \right)\left( {z_{n,1} - {{\mathbb{i}}\; z_{n,2}}} \right)}}} + {{\sum\limits_{n = 1}^{N}{y_{n,3}z_{n,3}}}}} \leq {\sqrt{\sum\limits_{n = 1}^{N}{\left( {y_{n,1}^{2} + y_{n,2}^{2}} \right){\sum\limits_{n = 1}^{N}\left( {z_{n,1}^{2} + z_{n,2}^{2}} \right)}}} + \sqrt{\sum\limits_{n = 1}^{N}{y_{n,3}^{2}{\sum\limits_{n = 1}^{N}z_{n,3}^{2}}}}}} = {{\sqrt{\left( {1 - {\sum\limits_{n = 1}^{N}y_{n,3}^{2}}} \right)\left( {1 - {\sum\limits_{n = 1}^{N}z_{n,3}^{2}}} \right)} + \sqrt{\sum\limits_{n = 1}^{N}{y_{n,3}^{2}{\sum\limits_{n = 1}^{N}z_{n,3}^{2}}}}} \leq 1}} & (32)\end{matrix}$

In (32) all the terms are equal if and only if for all n it is the casethat:y _(n,1) +iy _(n,2) =e ^(iτ)(z _(n,1) +iz _(n,2))y_(n,3)=z_(n,3)  (33)where τ is a fixed real number.

Equations (32) and (33) can be interpreted as a distance measure betweeny and z where all rotations are valid operations. The curves y and zhave distance 0 if and only if the projection curves in C and R havethis property.

Arbitrary unit vectors q=(q₁,q₂,q₃) in (31) can be treated in thefollowing way:

The matrix

$\begin{matrix}{Q = \begin{pmatrix}\frac{- q_{2}}{\sqrt{q_{1}^{2} + q_{2_{1}}^{2}}} & \frac{q_{1}}{\sqrt{q_{1}^{2} + q_{2}^{2}}} & 0 \\\frac{q_{1}q_{3}}{\sqrt{q_{1}^{2} + q_{2}^{2}}} & \frac{q_{2}q_{3}}{\sqrt{q_{1}^{2} + q_{2}^{2}}} & \frac{{- q_{1}^{2}} - q_{2}^{2}}{\sqrt{q_{1}^{2} + q_{2_{1}}^{2}}} \\q_{1} & q_{2} & q_{3}\end{pmatrix}} & (34)\end{matrix}$is orthogonal and maps q onto (0,0,1). In other words, it is sufficientto determine the distance between the curves Qy₁, . . . , Qy_(N) andQz₁, . . . , Qz_(N). According to (31) the new q is identical to(0,0,1).FIGS. 12A and 12B—Numerical Improvements and Alternatives

A number of possible improvements may be considered for some embodimentsof the above described methods. For example, for performance reasons itmay be desirable to replace the computation of full cross-correlationaccording to (25) with faster methods. Two such approaches areillustrated in FIGS. 12A and 12B, described below.

FIG. 12A—Scalar Product as Cross-Correlation Replacement

In some applications, one can further manipulate weight functions insuch a way that there is no need to compute a full cross-correlation. Infact, the calculation of one scalar product may be enough to generate avalid estimate for the best possible match. To achieve this goal thecorrect match may be encoded as phase information. FIG. 12A illustratesthe replacement of a full cross-correlation calculation for a curve witha simple scalar product.

Consider:

${{\sum\limits_{n = 1}^{N}{d_{n + k}d_{n}^{*}W_{n}}} = {{f_{n}{\exp\left( {2{\pi\mathbb{i}}\;{k/N}} \right)}\mspace{14mu}{for}\mspace{14mu} k} = 1}},\ldots\mspace{11mu},N$W_(n + 1) − W_(n) = 0  for  k = 1, …  , N

The constants f_(k) are chosen appropriately, e.g. 1 for k close to 1and 0.5 for all the other k.

FIG. 12B—Scalar Product as Cross-Correlation Replacement UsingLinearization of Angle

FIG. 12B illustrates a further development of the approach of FIG. 12A.A linearized angle may increase the accuracy of such a prediction, inthat an estimate will likely not be too far from the mark if the anglefunction is linear, i.e., has no flat regions. More specifically, thelinearization of angles may generate a one-to-one correspondence betweenangle of rotation and scalar product, and thus, the scalar product maybe used to determine the angle of rotation between two rotated versionsof a curve.

FIGS. 13-13B—Sub-Angle Accuracy

In some applications, it may be desirable to estimate small angularshifts between curves. For example, in geometric pattern matchingapplications, two discrete curves to be tested for a match may onlydiffer by a small rotation. Prior art methods have typically beenlimited to determining corrective rotations equal to or greater than theangular difference between successive points in the discrete curves,which may be considered as an intrinsic angular resolution of thecurves. In contrast, embodiments of the methods described herein mayoperate to determine corrective rotations of angles substantially lessthan these angular differences, hence the term “sub-angle accuracy”.Note that as used herein, the terms “angular shift” and “cyclic shift”refer to rotations, and that in this section, the term “shift” may alsobe used to refer to such rotations, where the fact that the shift is anangular or cyclic shift is assumed. The following describes one approachto that end.

Given a smooth continuous closed curve y and a second smooth curve z inthe complex plane of the same length (without loss of generality, letthe length be 1) such that y and z almost match, and where the mismatchis entirely based on slight rotational shifts, i.e., rotations, betweeny and z. In other words,z(τ)=y(τ+α)where α is a small but unknown real number and τ is the arc-length. Toestimate α, the term

Φ(α) = ∫₀¹𝕕τ[z(τ) − (y(τ) + α y^(′)(τ))][z(τ) − (y(τ) + α y^(′)(τ))]^(*)may be minimized, which results in

$\begin{matrix}{\alpha = \frac{\left( {\int_{0}^{1}{{\mathbb{d}{\tau\left\lbrack {{z(\tau)} - {y(\tau)}} \right\rbrack}}{y^{\prime}(\tau)}^{*}}} \right)}{\int_{0}^{1}{{\mathbb{d}\tau}{{y^{\prime}(\tau)}}^{2}}}} & (35)\end{matrix}$

where R denotes the real-valued portion of the expression. Formula (35)can be translated directly into expressions valid for discrete closedcurves y=(y₁, . . . , y_(N)) and z=(z₁, . . . , z_(N)) where the latteris a slightly rotated (rotationally shifted) version of the former.Assume both curves were sampled equidistantly. The discrete version ofthe estimate (35) is

$\begin{matrix}{\alpha = \frac{{\mathbb{e}}\left( {\sum\limits_{n = 1}^{N}{\left( {z_{n} - y_{n}} \right)\left( {y_{n + 1} - y_{n}} \right)^{*}}} \right)}{\sum\limits_{n = 1}^{N}{\left( {y_{n + 1} - y_{n}} \right)}^{2}}} & (36)\end{matrix}$

In (36) the value of α is normalized with respect to the index, i.e. avalue of 1 represents a shift or rotation of exactly 1 index point. Itis noted, however, that in general, the computed value of α will besubstantially less than 1, i.e., the value will generally be less thanthe intrinsic angular resolution of the discrete curves.

FIG. 13—Method for Determining Sub-Angle Accuracy

FIG. 13 is a high-level flowchart of one embodiment of a method forestimating a small shift a between two curves, y and z, where curve z isa slightly rotated version of curve y. Note that this method preferablyapplies to 2-dimensional discrete curves, where the curves have beennormalized with respect to position and length. It should be noted thatin various embodiments, some of the steps may be performed concurrently,in a different order than shown, or omitted. Additional steps may alsobe performed as desired.

As FIG. 13 shows, in 1302, α may be computed according to equation 36.

Then, in 1304, an interpolated (re-shifted or re-rotated) version of zmay be calculated using the computed value of α. It is noted thatvarious interpolation methods may be used. Quadratic or splineinterpolations generally preserve the original shape better than linearinterpolations and so may be used in preferred embodiments.

In 1306, a determination may be made as to whether further improvementsmay be made. If further improvements may be made, then the method mayreturn to 1302, calculating a new value of α using the interpolated(re-shifted or re-rotated) version of z, and continuing as describedabove. If no further improvements may be made, then the results, i.e.,α, may be output, as indicated in 1308.

Note that since the new value of α is based on a “corrected” version ofcurve z, the new value of α is preferably added to the original orcurrent value to produce an estimate of the rotational angle between thetwo original curves y and z, and thus, with each iteration the“cumulative” α may be updated. In other words, the resultant α is givenby:

$\alpha = {\sum\limits_{i}\alpha_{i}}$

where i is the iteration index, and α_(i) refers to each computed value.The results output in 1308 are preferably this cumulative α.

A more detailed embodiment of the method of FIG. 13 is described belowwith reference to FIG. 13A.

It should be noted that equations (35), (36), and the method of FIG. 13may be generalized to the case of spatial curves as well, as describedbelow with reference to FIG. 13B.

FIG. 13A—Method For Determining Sub-Angle Accuracy For 2-D DiscreteCurves

FIG. 13A is a more detailed flowchart of one embodiment of the method ofFIG. 13, i.e., a method for estimating a small rotational shift αbetween two 2D discrete curves, y and z, where curve z is a slightlyrotated version of curve y, and where the estimate may be used toimprove a match between the curves.

As FIG. 13A shows, in 1301, discrete curves y and z may be received, anda initialized to zero, where α refers to the cumulative rotational shiftmentioned above.

Then, in 1302, a current estimate of the rotational shift, referred toas α_(i), may be computed according to equation 36, where α_(i) is anestimate of the angle of rotation between y and z, and where i is aniteration index. Note that for iteration purposes, the curve z isdenoted by z^(i), indicating successive versions of the curve for eachiteration.

In 1303, α may be updated. More specifically, the current estimate ofthe rotational shift α_(i) may be used to update the cumulativerotational shift α (originally equal to zero) where

${\alpha = {\sum\limits_{i}\alpha_{i}}},$and α_(i) refers to each computed rotation estimate, as noted above.

Then, in 1304, a rotationally shifted version of z may be calculatedusing the computed value of α_(i). In other words, z^(i+1) may begenerated. As noted above, various interpolation methods may be used,but quadratic or spline interpolations generally preserve the originalshape better than linear interpolations and so may be used in preferredembodiments. Note that the new version of z may be computed by rotatingthe original curve z by the updated cumulative rotational shift α, oralternatively, may be computed by rotating the previous version of z bythe current estimate α_(i). The former approach is preferably used tominimize accumulated numerical errors.

In 1316, a determination may be made as to whether a stopping conditionoccurs. If a stopping condition is not detected, then the method mayreturn to 1302 and proceed iteratively, i.e., i=i+1, calculating a newvalue α_(i) using the current rotationally shifted version of z,performing the above steps in an iterative manner until the stoppingcondition obtains. Note that since the new value of α_(i) (i.e.,α_(i+1)) is based on the rotationally shifted version of z, themagnitude of the new value will generally be less than that of thecurrent (i.e., previous) value, and thus, as each successive computationof α_(i) is made, the values will tend toward zero, although theprogression may include both positive and negative values.

In one embodiment, the stopping condition may occur when the magnitudeof the computed value of α_(i) is less than some specified thresholdvalue (e.g., epsilon, representing a sufficient level of accuracy),typically after only a few iterations (possibly even just one).Alternatively, in another embodiment, the stopping condition may occurwhen a certain number of iterations have elapsed. Of course, any otherstopping condition may be used as desired.

If in 1316, a stopping condition is determined to occur, then theresults may be output as indicated in 1308, e.g., a final cumulativevalue of α may be output, e.g., to a display device, to a file, to aprocess, to an external system, and so forth, where α is an improvedestimate of the rotational difference between the two curves y and z.

In one embodiment, the most recent version of curve z (e.g., z^(k),where k is the index value of the last iteration) may be output, e.g.,to a matching process, where curves y and z^(k) may be analyzed todetermine whether or to what degree the curves match.

Is should be noted that equations (35), (36), and the method of FIG. 13may be generalized to the case of spatial curves, as described below.

Sub-Angle Accuracy in Spatial Curves

With some modification, the above sub-angle techniques may be applied tospatial curves, e.g., 3-dimensional discrete curves. Applications of thetechniques described herein are also contemplated for higher-dimensionalcurves, as well, e.g., discrete curves of four dimensions or higher, aswill be discussed further below.

Given a smooth continuous closed spatial curve y and a second smoothspatial curve z of same length (without loss of generality, let thelength be 1) such that y and z almost match where the mismatch isentirely based on slight rotations between y and z of the followingform:z(τ)=y(τ+α),where α is a small but unknown real number and τ is the arc-length. Anestimate of α is desired. To this end the term

Φ(α) = ∫₀¹𝕕τ{(z₁(τ) − (y₁(τ) + α y₁^(′)(τ)))² + (z₂(τ) − (y₂(τ) + α y₂^(′)(τ)))² + (z₃(τ) − (y₃(τ) + α y₃^(′)(τ)))²}is minimized (y and z contain three components each) which results in

$\begin{matrix}{\alpha = \frac{\int_{0}^{1}\ {{\mathbb{d}\tau}\begin{Bmatrix}{{\left( {{z_{1}(\tau)} - {y_{1}(\tau)}} \right){y_{1}^{\prime}(\tau)}} + \left( {{z_{2}(\tau)} -} \right.} \\{{\left. {y_{2}(\tau)} \right){y_{2}^{\prime}(\tau)}} + {\left( {{z_{3}(\tau)} - {y_{3}(\tau)}} \right){y_{3}^{\prime}(\tau)}}}\end{Bmatrix}}}{\int_{0}^{1}\ {{\mathbb{d}\tau}\left\{ {{y_{1}^{\prime}(\tau)}^{2} + {y_{2}^{\prime}(\tau)}^{2} + {y_{3}^{\prime}(\tau)}^{2}} \right\}}}} & (37)\end{matrix}$

Equation (37) may be directly translated into expressions valid for 3Ddiscrete closed curves y=((y₁₁,y₁₂,y₁₃), . . . , (y_(N1),y_(N2),y_(N3)))and z=((z₁₁,z₁₂,z₁₃), . . . , (z_(N1),z_(N2),z_(N3))) where the latteris a slightly rotated version of the former in the aforementioned sense.In a preferred embodiment, both curves are sampled substantiallyequidistantly. The discrete version of the estimate (37) is

$\begin{matrix}{\alpha = \frac{\sum\limits_{n = 1}^{N}\;\begin{Bmatrix}{{\left( {z_{n\; 1} - y_{n\; 1}} \right)\left( {y_{n + 11} - y_{n\; 1}} \right)} +} \\{{\left( {z_{n\; 2} - y_{n\; 2}} \right)\left( {y_{n + 12} - y_{n\; 2}} \right)} +} \\{\left( {z_{n\; 3} - y_{n\; 3}} \right)\left( {y_{n + 13} - y_{n\; 3}} \right)}\end{Bmatrix}}{\sum\limits_{n = 1}^{N}\left\{ {\left( {y_{n + 11} - y_{n\; 1}} \right)^{2} + \left( {y_{n + 12} - y_{n\; 2}} \right)^{2} + \left( {y_{n + 13} - y_{n\; 3}} \right)^{2}} \right\}}} & (38)\end{matrix}$

In one embodiment, the value of α in (38) may be normalized with respectto the index, i.e. a value of 1 may stand for a rotation or cyclic shiftof exactly 1 index point. It is noted, however, that in general, thecomputed value of α will be substantially less than 1, i.e., the valuewill generally be less than the intrinsic angular resolution of thediscrete curves.

FIG. 13B—Method For Determining Sub-Angle Accuracy For Spatial Curves

FIG. 13B is a high-level flowchart of a method for estimating a smallshift α between two spatial (i.e., 3D) curves, y and z, where curve z isa slightly rotated version of curve y, and where the estimate may beused to improve a match between the curves. Note that where the stepsare substantially the same as steps described above with reference toFIG. 13A, the descriptions may be abbreviated.

As FIG. 13B shows, in 1311, discrete spatial curves y and z may bereceived, and a initialized to zero.

Then, in 1312, a current estimate of the rotational shift, referred toas α_(i), may be computed according to equation 38, where α_(i) is anestimate of the angle of rotation between y and z, and where i is aniteration index. As noted above, for iteration purposes the curve z isdenoted by z^(i), indicating successive versions of the curve with eachiteration.

In 1303, α may be updated. More specifically, the current estimate ofthe rotational shift α_(i) may be used to update the cumulativerotational shift α (originally equal to zero) where

${\alpha = {\sum\limits_{i}\;\alpha_{i}}},$and α_(i) refers to each computed rotation estimate, as noted above.

Then, in 1314, a rotationally shifted version of the spatial curve z maybe calculated using the computed value of α_(i). In other words, z^(i+1)may be generated. In one embodiment, rotationally shifting the curve zmay include interpolating the curve. As noted above, quadratic or splineinterpolations are preferably used, although various other interpolationmethods may also be used as desired. As also noted above, the newversion of z may be computed by rotating the original curve z by theupdated cumulative rotational shift α, or alternatively, may be computedby rotating the previous version of z by the current estimate α_(i),although the former approach is preferably used to minimize accumulatednumerical errors.

In 1316, a determination may be made as to whether a stopping conditionoccurs. If a stopping condition is not detected, then the method mayreturn to 1322 and proceed iteratively, calculating a new value of α_(i)(i.e., α_(i+1)) using the current rotationally shifted version of z, andperforming the above steps in an iterative manner until the stoppingcondition obtains.

As described above, in one embodiment, the stopping condition may occurwhen the computed value of α_(i) is less than some specified thresholdvalue (e.g., epsilon, representing a sufficient level of accuracy), or,alternatively, the stopping condition may occur when a certain number ofiterations have elapsed. Any other stopping condition may also bespecified as desired.

If in 1316 a stopping condition is determined to occur, then the resultsmay be output, as indicated in 1308, e.g., a final cumulative value of αmay be output, e.g., to a display device, to a file, to a process, to anexternal system, and so forth, where α is an improved estimate of therotational difference between the two curves y and z.

In one embodiment, the most recent version of curve z (e.g., z^(k),where k is the index value of the last iteration) may be output, e.g.,to a matching process, where curves y and z^(k) may be analyzed todetermine whether or to what degree the curves match, as describedabove.

Thus, various embodiments of the above methods may be used to determinesub-angle accuracy for two- and three-dimensional discrete curves (oreven higher-dimensional curves), where a second curves is a slightlyrotated version of a first curve. Once the second curve has been rotatedin accordance with the computed value of α, the curves may be tested orprocessed to determine whether and/or to what extent they match, asdescribed above.

Applications of the Spatial Curve Sub-Angle Technique

The method of FIG. 13B is applicable to many types of problems where 3Dgeometric pattern matching is involved (i.e., 3D discrete curvematching). Some examples include quality control in automatedmanufacturing where 3D parts may be checked for proper orientation,topographic analysis and/or matching of textured paintings, e.g., todetect counterfeit works, and biometric analysis, e.g., face-matching,among others. For example, two-camera systems or scanners augmented withsensors may generate 3- or higher dimensional data, i.e., curves.

Artificial 3D Curves

In addition to these standard 3D applications, there are other, lessobvious, applications. For example, in one application of the method ofFIG. 13B, 2D curves may be augmented with additional information relatedto the curves, e.g., gray-scale information, to improve the accuracy ingeometric pattern matching. In other words, gray-scale information maybe used as an additional dimension, effectively making the 2D curvesinto 3D curves. To this end the shape information (two spatialcoordinates in the 2-D case) may be augmented by the gray-scaleinformation (a third spatial coordinate in the 2-D case) at each point.Thus, the gray-scale information may be interpreted as an extradimension in the curves. The method of FIG. 13B may then be used todetermine sub-angle rotational differences between two (originally 2D)curves, where the additional information may serve to improve theaccuracy of subsequent curve matching processes.

For example, in one embodiment, the spatial curve approach describedabove with reference to FIG. 13B may be used in 2D geometric patternmatching in the following manner: First, the 2D discrete curve may berecognized or represented as sequence of (x,y) coordinates. Insituations where the curve is derived from an image, this is generallyperformed based on edge detection, where any gray-scale information isusually lost. In other words, the image is usually processed to maximizecontrast, converting the image to a binary (black and white) image,where sharp transitions determine the edge that constitutes the curve,and where any gray-scale information is discarded.

In contrast, in embodiments of the present approach, the gray-scaleinformation is not discarded, but instead may be re-introduced, i.e., tothe curves as an extra dimension. It is noted that a sharp transitioncan be based on completely different gray-scale values, in that edgesare determined by gradients, e.g., by relative differences, rather thanby absolute values, and so in an alternative embodiment, the edgedetection may be performed in the gray-scale domain. Thus, for eachpoint in the curves, a number representing the gray-scale value may beadded to the existing (x,y) coordinates, resulting in an “artificial”spatial curve (x,y, gray-scale).

Now, the method of FIG. 13B may be applied as describe above, where theformerly 2D curve has been re-cast as an “artificial” 3D curve. Forexample, the second curve, e.g., curve z, may be re-rotated a smallangle alpha, e.g., each point in the curve may be rotated through thesmall angle alpha, (x(t+alpha),y(t+alpha),gray-scale(t+alpha)), where‘t’ refers to the original angular position or arc length of the point(using an inner coordinate system).

EXAMPLE APPLICATION Wafer Manufacturing and Quality Control

In an automated manufacturing facility where semiconductor wafers areproduced, consider an image of a wafer (circle on background) wherethere are notches on the perimeter of the wafer. The notches may beencoded or represented by gray-scales, e.g., the notches are darker orbrighter than the background. Classical geometric pattern matchingapproaches (for slightly rotated curves) are not adequate because acircle is radially symmetric, and so rotations of the (black and white)2D image are all degenerate. However, using the above approach, thegray-scale information may be included as a third dimension, in whichcase the image may be rotated to attempt to align the notches, and sofacilitate pattern matching, e.g., for quality control.

It should be noted that this approach is also applicable in higherdimensions, where, for example, other additional image data may beincluded as corresponding other dimensions to be added to curves thatare originally 2D or 3D (or even higher dimensionality). For example,where the first discrete curve and the second discrete curve eachcomprise a respective sequence of points, each point may include arespective two or more spatial coordinates, and a respective one or moreadditional coordinates, where each additional coordinate is based onrespective additional image data associated with the point. Each of theone or more additional coordinates may then be interpreted as additionalspatial coordinates for the point. Thus, the effective dimensionality ofthe curves may be increased by including additional image data for eachpoint as additional spatial coordinates or dimensions (for example: 3color dimensions).

FIGS. 14A and 14B—Alternative 2D Matching Method

Let y=(y₁, . . . , y_(N)) and z=(z₁, . . . , z_(N)) be two discreteclosed curves in the complex plane. Let s be a fixed natural number.Generally, s will have a value of 1, however, in various embodiments,other choices are possible and even desirable from a numericalstandpoint. Let y and z be normalized in the following sense:

$\begin{matrix}{{\sum\limits_{n = 1}^{N}\;{{y_{n + s} - y_{n}}}^{2}} = {{\sum\limits_{n = 1}^{N}\;{{z_{n + s} - z_{n}}}^{2}} = 1}} & (37)\end{matrix}$where the index notation is understood cyclically. It is the case that

${\sum\limits_{n = 1}^{N}\;\left( {y_{n + s} - y_{n}} \right)} = {{\sum\limits_{n = 1}^{N}\;\left( {z_{n + s} - z_{n}} \right)} = 0}$

In (38) a complex number is defined where the magnitude is less than 1unless z is an exact rotated version of y (Cauchy-Hoelder-Schwarzinequality).

$\begin{matrix}{\sum\limits_{n = 1}^{N}\;{\left( {y_{n + s} - y_{n}} \right)\;\left( {z_{n + s} - z_{n}} \right)^{*}}} & (38)\end{matrix}$

(38) generates a new class of curve matching algorithms. Theorem 1,described above, uses complex vectors y and z to match discrete closedcurves. According to (38) many other approaches are valid and may offernumerical advantages. In particular, the choice s=N/2 may reduce thecomputational load in cases where the y- and z-components are equallydistributed.

The following method matches two discrete closed planar curves y and zof size N in the complex plane where shift, scaling, and rotation arevalid operations.

FIG. 14A—Matching 2D Discrete Curves

FIG. 14A flowcharts one embodiment of a method for matching two 2Ddiscrete curves, y and z. As noted above, in various embodiments, someof the steps may be performed in a different order than shown, or may beomitted. Additional steps may also be performed.

As FIG. 14A shows, in 1402, an appropriate natural number s may beselected from [1 . . . N−1], where N is equal to the (desired) length ornumber of points in the discrete curves, y and z.

In 1404, the two discrete curves, y and z, may be normalized, inaccordance with equation (37) above. In other words, the normalizedcurves have N 2D points, and the lengths satisfy equation (37).

In 1406, all complex numbers

$\sum\limits_{n = 1}^{N}\;{\left( {y_{n + s + k} - y_{n + k}} \right)\;\left( {z_{n + s} - z_{n}} \right)^{*}}$may be calculated for k=1, . . . , N.

In 1408, the values of k for which p approaches unity may be determinedto indicate matches. In other words, if any of the calculated complexnumbers has a magnitude close to 1, a match has been found.

Finally, in 606, the pattern match results may be output.

FIG. 14B—Illustration of 2D Curve Matching

FIG. 14B illustrates the results of applying the method of FIG. 14A to aspecific closed curve. The triangle's cross-correlation (upper curve ofthe lower graph) is less robust than the new one (lower curve) resultingfrom the method of FIG. 14A. In this case, s=1. The circles in the upperright part of FIG. 14B mark cluster points of the new representation ofthe triangle. These cluster points represent the sides of the triangle.For small s the use of filter operations applied to the original closedcurve is recommended. It is noted that all points of the new curve areconnected by lines, and that the points have an almost constant distancefrom the center.

It may also be possible to apply schemes used in (25) and the method ofFIG. 9B to further improve the quality of matching results. In this casethe similarity measure (38) must be replaced with more general measuressuch as

$\begin{matrix}{\sum\limits_{i = 1}^{N}\;{\left( {y_{i + s} - y_{i}} \right)\;\left( {z_{i + s} - z_{i}} \right)^{*}W_{i}}} & (39)\end{matrix}$where W is a complex weight function.

A second method may be based on the simple observation that (y₁y₁*, . .. , y_(N)y_(N)*) and (z₁z₁*, . . . , z_(N)z_(N)*) can be regarded asreal-valued versions of the original discrete and normalized curves yand z (sum of norms is equal to 1). In other words, matching of discretecurves in the complex plane can be translated into matching of discretecurves in the real number line. The latter is faster and animplementation is more straightforward. This approach is valid becauseof the following observation. A perfect matching of (y₁y₁*, . . . ,y_(N)y_(N)*) and (z₁z₁*, . . . , z_(N)z_(N)*) is achievable, if and onlyif these two sequences are exactly the same. Furthermore, the discretecurves y and z are equally sampled and have the same orientation in thecomplex plane. It therefore follows that the discrete curves y and zcoincide in the complex plane.

FIG. 15—Matching 3D Closed Curves

FIG. 15 is a flowchart of one embodiment of a method for matching 3Ddiscrete closed curves. More specifically, the method matches twodiscrete closed spatial curves y and z of size N where shift, scaling,and rotation are valid operations. As previously stated, in variousembodiments, some of the steps may be performed in a different orderthan shown, or may be omitted, as desired. Additional steps may also beperformed.

As FIG. 15 shows, in 1502, two 3D closed discrete curves, y and z, maybe normalized, using any of the normalization schemes mentioned above.After normalization, each curve is of length N. It should be noted thatin one embodiment, a determination may be made as to whether y and z areidentical (or almost a match) before proceeding with the method because,as mentioned above, the algorithm is not reliable for small or zeroangles of rotation. If the curves do not already match (or almostmatch), then the method continues.

A unit rotation axis vector q may then be computed based on equation(31), as indicated in 1504. Then, in 1506, a determination may be madeas to whether q=(0,0,1), i.e., whether q is equal to the z-axis. Ifq=(0,0,1), then the method continues with step 1514 below.

If q is not equal to (0,0,1), then an orthonormal matrix Q may becomputed based on q, in accordance with equation (34), as indicated by1508. Then, in 1510, y (or y′)=Qy and z=Qz may be computed. In otherwords, each curve may be multiplied by the matrix Q.

In 1512, the expression:

$\begin{matrix}{p = {{{\sum\limits_{n = 1}^{N}\;{\left( {y_{n,1} + {i\; y_{n,2}}} \right)\left( {z_{n,1} - {i\; z_{n,2}}} \right)}}} + {\sum\limits_{n = 1}^{N}\;{y_{n,3}z_{n,3}}}}} & (40)\end{matrix}$

may be calculated where n refers to a particular point in the curve, andthe second subscripts indicate the first, second, and third componentsof the data point, i.e., the x, y, and z components.

In 1514, the value of p computed in 1512 may be compared to unity todetermine whether the curves y and z match, where y and z match if andonly if p is close to 1.

Finally, in 606, the pattern matching results may be output, asdescribed above. It should be noted that if one is to perform matchingbetween y and all cyclically shifted versions of z (or between z and allcyclically shifted versions of y), the above steps must be performed Ntimes.

Higher Dimensional Curves

As noted above, in some embodiments, the above techniques may be appliedto higher dimensional curves, i.e., curves of 4 or more dimensions. Onecomplication with higher dimensional rotations is the number oforthogonal axes around which rotations may occur. For example, in 2dimensions there is only one rotational axis (normal to the plane), in3D there are 3 orthogonal axes, in 4D there are 6 orthogonal axes. Moregenerally, in n dimensions there are n(n−1)/2 orthogonal axes. The Liegroup of rotations in nD is referred to as SO(n).

In some embodiments of the present invention, the rotational axis (whichmay correspond to a composition of rotations around the orthogonal axes)may be known. Thus, given the rotational axis, α, which, as describedabove, is an estimate of a small angle between two higher-dimensionalcurves (first and second discrete curves y and z), may be determinedthusly:

$\begin{matrix}{{\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},} & (41)\end{matrix}$

where n indexes a respective point in the curves (of a total of Npoints), and d denotes dimension (with maximum D). Note that equation 41corresponds to equation (38), but applies to higher dimensional curves,e.g., discrete closed curves in 4 or more dimensions.

Thus, equation (41) may thus be used to estimate a (small) rotationalshift between the first and second discrete curves in 4 or moredimensions. Moreover, as explained above in detail, the estimate may beused to perform curve matching in any of a variety of applications.

Applications

Two typical applications related to machine vision that may besuccessfully solved with methods described herein are now described.

Geometric Pattern Matching Based On Edges

In geometric pattern matching, e.g., discrete curve matching, a knowntemplate may be characterized geometrically (based on shape). It iscommon to use edge information. From a real-time prospective, theso-called learning phase is less demanding and lists of efficientdescriptors of detected curves can be built up carefully. During thematching process itself the template may be searched for in a continuous(real-time) stream of images. In many cases the actual template is ashifted, rotated, and/or scaled version of the original template.

Optical Character Recognition (OCR)

Various embodiments of the method presented herein are particularlysuited for optical character recognition. More specifically, the mappingof standard alphanumeric characters to forms more easily discriminatedusing a computed weighting function or mapping operator may greatlyimprove the reliability of OCR, such as may be used in optical scannersand reading machines for the visually impaired, among othersapplications.

Further exemplary applications for the above described techniquesapplied to curves in various dimensions may include, but are not limitedto, protein folding, image processing, and robotics, among others. Forexample, in a protein folding application, the candidate curve may bechecked (e.g., pattern-matched) against a database of existing proteins,which may be expressed as a 3D problem, or if generalized coordinatesare used, may be expressed as a higher-dimensional problem. Imageprocessing may be addressed as a 3D problem (as described above), or, incases where a curve is checked against 2 or 3 curves in parallel, orwhen additional data are included in the image(s), may generatehigher-dimensional problems, e.g., 6D, 9D, etc. This also applies to 2Dimage processing with 2 or more curves. In robotics applications, whichmay involve paths in configuration spaces, dimensionalities in the500-1000 range (or higher) may be common.

More generally, many systems and processes may be represented,described, characterized, or analyzed using generalized coordinates ofhigh-dimensionality. Note that these systems and processes may bephysical, e.g., machines, manufacturing processes, etc., or may belogical or abstract, e.g., large database operations, e.g.,high-dimensional searches, network communication/connectivity analysis,etc., where, for example, the discrete curves may represent trajectoriesin an abstract configuration space regarding abstract concepts ormechanisms, e.g., steps in a process, connections in a communicationpath, decision nodes in a decision strategy or expert system, logicalsteps in a proof, and so forth.

Thus, embodiments of the above techniques may thus be used in any of aplethora of applications.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

1. A computer-accessible memory medium that stores program instructionsfor estimating a rotational shift between a first discrete curve and asecond discrete curve, wherein the program instructions are executableto perform: receiving a first discrete curve and a second discretecurve, wherein the second discrete curve is a rotationally shiftedversion of the first discrete curve, wherein the first discrete curveand the second discrete curve each comprises a respective curve in atleast four dimensions; estimating a rotational shift between the firstdiscrete curve and the second discrete curve based on the first discretecurve and the second discrete curve, wherein said estimating therotational shift between the first discrete curve and the seconddiscrete curve based on the first discrete curve and the second discretecurve comprises calculating:${\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},$wherein y and z refer respectively to points in the first discrete curveand the second discrete curve, indexed by n and d, wherein n denotes aparticular point of N points, and d denotes dimension up to a maximum ofD; updating a cumulative rotational shift based on the estimatedrotational shift; generating a rotationally shifted version of thesecond discrete curve based on the cumulative rotational shift;performing said estimating, said updating, and said generating in aniterative manner using the respective rotationally shifted discretecurve for each iteration until a stopping condition occurs, therebydetermining a final estimate of the rotational shift between the firstdiscrete curve and the second discrete curve; and storing the finalestimate of the rotational shift, wherein the final estimate of therotational shift is useable to perform discrete curve matching.
 2. Thememory medium of claim 1, wherein each point of the first and seconddiscrete curves comprises generalized coordinates.
 3. The memory mediumof claim 1, wherein the first discrete curve and the second discretecurve each comprises a respective spatial curve.
 4. The memory medium ofclaim 1, wherein each point of the first and second curves comprises: atleast three 3D spatial coordinates; and a respective one or moreadditional coordinates, each based on respective additional dataassociated with the point, wherein the one or more additionalcoordinates are interpreted as additional spatial coordinates for thepoint.
 5. The memory medium of claim 1, wherein the first discrete curveand the second discrete curve each comprises a respective sequence ofpoints, wherein each point comprises: a respective two or more spatialcoordinates; and a respective two or more additional coordinates, eachbased on respective additional data associated with the point, whereinthe one or more additional coordinates are interpreted as additionalspatial coordinates for the point.
 6. The memory medium of claim 5,wherein the respective two or more additional coordinates comprisesimage data, comprising one or more of: a gray-scale value, wherein thegray-scale value is interpreted as another spatial coordinate; or one ormore color values, wherein the one or more color values are interpretedas other spatial coordinates.
 7. The memory medium of claim 1, whereinthe stopping condition occurs when a magnitude of the estimatedrotational shift is less than a specified threshold value.
 8. The memorymedium of claim 1, wherein the stopping condition occurs when aspecified number of iterations have elapsed.
 9. The memory medium ofclaim 1, wherein the first discrete curve and the second discrete curveeach comprise a respective sequence of points in a complex space. 10.The memory medium of claim 1, wherein the first discrete curve and thesecond discrete curve comprise substantially smooth discrete curves. 11.The memory medium of claim 1, wherein the first discrete curve and thesecond discrete curve each comprises a respective closed curve.
 12. Thememory medium of claim 1, wherein the first discrete curve and thesecond discrete curve each comprises the same number of points, whereinfor each curve, each point corresponds to a value of an index; andwherein the value of the estimated rotational shift is normalized withrespect to the index.
 13. The memory medium of claim 12, wherein anestimated a rotational shift value of 1 corresponds to a cyclic shift ofone point in the second discrete curve.
 14. The memory medium of claim1, wherein the second discrete curve is rotationally shifted from thefirst discrete curve by less than a rotational distance betweensuccessive points in the second discrete curve.
 15. The memory medium ofclaim 1, wherein the second discrete curve is rotationally shifted fromthe first discrete curve by less than or equal to 5 degrees.
 16. Thememory medium of claim 1, wherein the second discrete curve isrotationally shifted from the first discrete curve by less than or equalto 1 degree.
 17. The memory medium of claim 1, wherein said rotationalshifting the second discrete curve based on the estimated rotationalshift to generate a respective rotationally shifted discrete curvefurther comprises: interpolating the second discrete curve.
 18. Thememory medium of claim 1, wherein the program instructions are furtherexecutable to perform: outputting the final estimate of the rotationalshift between the first discrete curve and the second discrete curve.19. The memory medium of claim 1, wherein the program instructions arefurther executable to perform: performing curve matching on the firstdiscrete curve and the second discrete curve using the final estimate ofthe rotational shift, thereby generating results indicating a degree ofmatch between the first discrete curve and the second discrete curve.20. A method for estimating a rotational shift between a first discretecurve and a second discrete curve, the method comprising: receiving afirst discrete curve and a second discrete curve, wherein the seconddiscrete curve is a rotationally shifted version of the first discretecurve; estimating a rotational shift between the first discrete curveand the second discrete curve based on the first discrete curve and thesecond discrete curve, wherein said estimating the rotational shiftbetween the first discrete curve and the second discrete curve based onthe first discrete curve and the second discrete curve comprisescalculating:${\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},$wherein y and z refer respectively to points in the first discrete curveand the second discrete curve, indexed by n and d, wherein n denotes aparticular point of N points, and d denotes dimension up to a maximum ofD; updating a cumulative rotational shift based on the estimatedrotational shift; generating a rotationally shifted version of thesecond discrete curve based on the cumulative rotational shift; andperforming said estimating, said updating, and said generating in aniterative manner using the respective rotationally shifted discretecurve for each iteration until a stopping condition occurs, therebydetermining a final estimate of the rotational shift between the firstdiscrete curve and the second discrete curve; and storing the finalestimate of the rotational shift, wherein the final estimate of therotational shift is useable to perform discrete curve matching.
 21. Asystem for estimating a rotational shift between a first discrete curveand a second discrete curve, the system comprising: a processor; amemory medium coupled to the processor; an input coupled to theprocessor and the memory medium; and an output coupled to the processorand the memory medium; wherein the input is operable to receive a firstdiscrete curve and a second discrete curve, wherein the second discretecurve is a rotationally shifted version of the first discrete curve, andwherein the first discrete curve and the second discrete curve eachcomprises a respective sequence of points in at least three dimensions;wherein the memory medium stores program instructions which areexecutable by the processor to: estimate a rotational shift between thefirst discrete curve and the second discrete curve based on the firstdiscrete curve and the second discrete curve, wherein said estimatingthe rotational shift between the first discrete curve and the seconddiscrete curve based on the first discrete curve and the second discretecurve comprises calculating:${\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},$wherein y and z refer respectively to points in the first discrete curveand the second discrete curve, indexed by n and d, wherein n denotes aparticular point of N points, and d denotes dimension up to a maximum ofD; update a cumulative rotational shift based on the estimatedrotational shift; generate a rotationally shifted version of the seconddiscrete curve based on the cumulative rotational shift; and performsaid estimating, said updating, and said generating in an iterativemanner using the respective rotationally shifted discrete curve for eachiteration until a stopping condition occurs, thereby determining a finalestimate of the rotational shift between the first discrete curve andthe second discrete curve; wherein the output is operable to output thefinal estimate of the rotational shift between the first discrete curveand the second discrete curve, and wherein the final estimate of therotational shift is useable to perform discrete curve matching.
 22. Asystem for estimating a rotational shift between a first discrete curveand a second discrete curve, the system comprising: means for receivinga first discrete curve and a second discrete curve, wherein the seconddiscrete curve is a rotationally shifted version of the first discretecurve, wherein the first discrete curve and the second discrete curveeach comprises a respective curve in at least three dimensions; meansfor estimating a rotational shift between the first discrete curve andthe second discrete curve based on the first discrete curve and thesecond discrete curve, wherein said estimating the rotational shiftbetween the first discrete curve and the second discrete curve based onthe first discrete curve and the second discrete curve comprisescalculating:${\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},$wherein y and z refer respectively to points in the first discrete curveand the second discrete curve, indexed by n and d, wherein n denotes aparticular point of N points, and d denotes dimension up to a maximum ofD; means for updating a cumulative rotational shift based on theestimated rotational shift; means for generating a rotationally shiftedversion of the second discrete curve based on the cumulative rotationalshift; means for performing said estimating, said updating, and saidgenerating in an iterative manner using the respective rotationallyshifted discrete curve for each iteration until a stopping conditionoccurs, thereby determining a final estimate of the rotational shiftbetween the first discrete curve and the second discrete curve; andmeans for storing the final estimate of the rotational shift, whereinthe final estimate of the rotational shift is useable to performdiscrete curve matching.
 23. A programmable hardware element configuredfor estimating a rotational shift between a first discrete curve and asecond discrete curve, wherein the programmable hardware element isconfigured to perform: receiving a first discrete curve and a seconddiscrete curve, wherein the second discrete curve is a rotationallyshifted version of the first discrete curve, wherein the first discretecurve and the second discrete curve each comprises a respective curve inat least three dimensions; estimating a rotational shift between thefirst discrete curve and the second discrete curve based on the firstdiscrete curve and the second discrete curve, wherein said estimatingthe rotational shift between the first discrete curve and the seconddiscrete curve based on the first discrete curve and the second discretecurve comprises calculating:${\alpha = \frac{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;{\left( {z_{nd} - y_{nd}} \right)\left( {y_{{({n + 1})}d} - y_{nd}} \right)}}}{\sum\limits_{n = 1}^{N}\;{\sum\limits_{d = 1}^{D}\;\left( {y_{{({n + 1})}d} - y_{nd}} \right)^{2}}}},$wherein y and z refer respectively to points in the first discrete curveand the second discrete curve, indexed by n and d, wherein n denotes aparticular point of N points, and d denotes dimension up to a maximum ofD; updating a cumulative rotational shift based on the estimatedrotational shift; generating a rotationally shifted version of thesecond discrete curve based on the cumulative rotational shift;performing said estimating, said updating, and said generating in aniterative manner using the respective rotationally shifted discretecurve for each iteration until a stopping condition occurs, therebydetermining a final estimate of the rotational shift between the firstdiscrete curve and the second discrete curve; and storing the finalestimate of the rotational shift, wherein the final estimate of therotational shift is useable to perform discrete curve matching.